Jul 16 15:33:27.435562 ip-10-0-134-168 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 16 15:33:27.435573 ip-10-0-134-168 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 16 15:33:27.435582 ip-10-0-134-168 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 16 15:33:27.435873 ip-10-0-134-168 systemd[1]: Failed to start Kubernetes Kubelet. Jul 16 15:33:37.488550 ip-10-0-134-168 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 16 15:33:37.488568 ip-10-0-134-168 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 0922598dc1a84494a5db13cf642ead82 -- Jul 16 15:35:12.170907 ip-10-0-134-168 systemd[1]: Starting Kubernetes Kubelet... Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.621770 2553 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629348 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629359 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629362 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629365 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629368 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629371 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629374 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:12.637258 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629377 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629380 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629383 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629386 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629389 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629391 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629394 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629397 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629400 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629403 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629406 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629408 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629416 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629419 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629422 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629425 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629427 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629431 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629433 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629436 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:12.703211 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629438 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629441 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629444 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629447 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629449 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629451 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629454 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629457 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629459 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629462 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629464 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629467 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629469 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629472 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629474 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629477 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629480 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629483 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629486 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629490 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:12.709181 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629493 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:12.706168 ip-10-0-134-168 systemd[1]: Started Kubernetes Kubelet. Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629495 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629498 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629501 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629504 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629507 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629509 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629512 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629514 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629517 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629521 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629524 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629528 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629531 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629533 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629536 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629539 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629541 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629544 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629547 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:12.716379 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629549 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629552 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629555 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629557 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629560 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629562 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629565 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629568 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629571 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629573 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629576 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629579 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629581 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629584 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629586 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629589 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629591 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629594 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629598 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:12.720488 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629601 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629604 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629607 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629610 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629613 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629616 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629631 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629634 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629638 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629642 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629645 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629648 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.629650 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631265 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631272 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631276 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631279 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631282 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631285 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:12.723668 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631288 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631291 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631293 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631296 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631299 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631302 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631305 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631307 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631310 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631313 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631316 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631318 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631321 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631324 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631326 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631329 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631331 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631334 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631337 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631340 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:12.729816 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631345 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631349 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631352 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631354 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631357 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631359 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631362 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631365 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631367 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631370 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631373 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631375 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631378 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631381 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631383 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631386 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631389 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631391 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631395 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631397 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:12.733903 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631400 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631402 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631405 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631407 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631410 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631414 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631417 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631420 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631423 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631426 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631430 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631433 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631435 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631438 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631441 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631443 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631446 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631448 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631450 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:12.739850 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631453 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631456 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631458 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631461 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631464 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631466 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631469 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631471 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631474 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631477 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631479 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631482 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631485 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631487 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631490 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631492 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631495 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631498 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631501 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631503 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:12.887551 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631506 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631509 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631511 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631514 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631517 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631520 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631522 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631526 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631529 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631532 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631537 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631540 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631543 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631546 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631924 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631929 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631933 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631936 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631939 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:12.906473 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631941 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631946 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631950 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631953 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631957 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631960 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631963 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631966 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631969 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631972 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631975 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631978 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631981 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631983 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631986 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631989 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631991 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631994 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.631997 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:12.908611 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632000 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632002 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632005 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632007 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632013 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632015 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632018 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632021 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632023 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632026 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632028 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632031 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632034 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632037 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632039 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632042 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632044 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632047 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632051 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:12.910226 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632055 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632057 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632060 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632063 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632066 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632070 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632073 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632076 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632078 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632080 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632083 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632086 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632088 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632091 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632093 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632096 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632099 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632102 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632105 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632107 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:12.912729 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632110 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632112 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632115 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632117 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632120 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632122 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632125 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632128 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632130 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632133 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632136 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632138 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632140 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632143 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632146 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632148 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632151 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632155 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632158 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632160 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:12.914819 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632163 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632166 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632169 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632171 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632174 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632176 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632179 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632181 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632184 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632188 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632190 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632193 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632195 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632198 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632200 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.632203 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632275 2553 flags.go:64] FLAG: --address="0.0.0.0" Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632285 2553 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632294 2553 flags.go:64] FLAG: --anonymous-auth="true" Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632300 2553 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632306 2553 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 16 15:35:12.916973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632311 2553 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632317 2553 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632322 2553 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632325 2553 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632328 2553 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632332 2553 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632335 2553 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632338 2553 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632343 2553 flags.go:64] FLAG: --cgroup-root="" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632345 2553 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632348 2553 flags.go:64] FLAG: --client-ca-file="" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632360 2553 flags.go:64] FLAG: --cloud-provider="external" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632363 2553 flags.go:64] FLAG: --cluster-dns="[]" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632367 2553 flags.go:64] FLAG: --cluster-domain="" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632370 2553 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632373 2553 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632376 2553 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632379 2553 flags.go:64] FLAG: --container-log-max-files="5" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632383 2553 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632386 2553 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632391 2553 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632394 2553 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632397 2553 flags.go:64] FLAG: --contention-profiling="false" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632400 2553 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 16 15:35:12.917780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632403 2553 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632406 2553 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632409 2553 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632415 2553 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632418 2553 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632421 2553 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632424 2553 flags.go:64] FLAG: --enable-load-reader="false" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632427 2553 flags.go:64] FLAG: --enable-server="true" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632431 2553 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632436 2553 flags.go:64] FLAG: --event-burst="100" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632439 2553 flags.go:64] FLAG: --event-qps="50" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632442 2553 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632444 2553 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632447 2553 flags.go:64] FLAG: --eviction-hard="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632451 2553 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632454 2553 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632458 2553 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632461 2553 flags.go:64] FLAG: --eviction-soft="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632464 2553 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632467 2553 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632470 2553 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632473 2553 flags.go:64] FLAG: --experimental-mounter-path="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632475 2553 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632478 2553 flags.go:64] FLAG: --fail-swap-on="true" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632481 2553 flags.go:64] FLAG: --feature-gates="" Jul 16 15:35:12.919406 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632485 2553 flags.go:64] FLAG: --file-check-frequency="20s" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632488 2553 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632491 2553 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632495 2553 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632498 2553 flags.go:64] FLAG: --healthz-port="10248" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632501 2553 flags.go:64] FLAG: --help="false" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632504 2553 flags.go:64] FLAG: --hostname-override="ip-10-0-134-168.ec2.internal" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632507 2553 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632510 2553 flags.go:64] FLAG: --http-check-frequency="20s" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632513 2553 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632516 2553 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632519 2553 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632522 2553 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632525 2553 flags.go:64] FLAG: --image-service-endpoint="" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632527 2553 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632530 2553 flags.go:64] FLAG: --kube-api-burst="100" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632533 2553 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632536 2553 flags.go:64] FLAG: --kube-api-qps="50" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632539 2553 flags.go:64] FLAG: --kube-reserved="" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632542 2553 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632544 2553 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632547 2553 flags.go:64] FLAG: --kubelet-cgroups="" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632551 2553 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632555 2553 flags.go:64] FLAG: --lock-file="" Jul 16 15:35:12.921027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632558 2553 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632561 2553 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632564 2553 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632569 2553 flags.go:64] FLAG: --log-json-split-stream="false" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632572 2553 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632574 2553 flags.go:64] FLAG: --log-text-split-stream="false" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632577 2553 flags.go:64] FLAG: --logging-format="text" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632580 2553 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632583 2553 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632586 2553 flags.go:64] FLAG: --manifest-url="" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632589 2553 flags.go:64] FLAG: --manifest-url-header="" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632594 2553 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632597 2553 flags.go:64] FLAG: --max-open-files="1000000" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632601 2553 flags.go:64] FLAG: --max-pods="110" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632604 2553 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632607 2553 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632610 2553 flags.go:64] FLAG: --memory-manager-policy="None" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632613 2553 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632615 2553 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632632 2553 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632635 2553 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632641 2553 flags.go:64] FLAG: --node-status-max-images="50" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632644 2553 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632647 2553 flags.go:64] FLAG: --oom-score-adj="-999" Jul 16 15:35:12.921858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632651 2553 flags.go:64] FLAG: --pod-cidr="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632654 2553 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632659 2553 flags.go:64] FLAG: --pod-manifest-path="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632662 2553 flags.go:64] FLAG: --pod-max-pids="-1" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632665 2553 flags.go:64] FLAG: --pods-per-core="0" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632668 2553 flags.go:64] FLAG: --port="10250" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632671 2553 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632674 2553 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-00f838ab2268e38a5" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632678 2553 flags.go:64] FLAG: --qos-reserved="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632681 2553 flags.go:64] FLAG: --read-only-port="10255" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632684 2553 flags.go:64] FLAG: --register-node="true" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632687 2553 flags.go:64] FLAG: --register-with-taints="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632690 2553 flags.go:64] FLAG: --registry-burst="10" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632693 2553 flags.go:64] FLAG: --registry-qps="5" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632696 2553 flags.go:64] FLAG: --reserved-cpus="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632699 2553 flags.go:64] FLAG: --reserved-memory="" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632702 2553 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632705 2553 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632708 2553 flags.go:64] FLAG: --rotate-certificates="false" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632712 2553 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632715 2553 flags.go:64] FLAG: --runonce="false" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632718 2553 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632721 2553 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632724 2553 flags.go:64] FLAG: --seccomp-default="false" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632727 2553 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 16 15:35:12.922673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632730 2553 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632732 2553 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632735 2553 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632738 2553 flags.go:64] FLAG: --storage-driver-password="root" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632741 2553 flags.go:64] FLAG: --storage-driver-secure="false" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632744 2553 flags.go:64] FLAG: --storage-driver-table="stats" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632747 2553 flags.go:64] FLAG: --storage-driver-user="root" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632750 2553 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632753 2553 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632756 2553 flags.go:64] FLAG: --system-cgroups="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632759 2553 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632764 2553 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632767 2553 flags.go:64] FLAG: --tls-cert-file="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632770 2553 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632774 2553 flags.go:64] FLAG: --tls-min-version="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632778 2553 flags.go:64] FLAG: --tls-private-key-file="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632781 2553 flags.go:64] FLAG: --topology-manager-policy="none" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632800 2553 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632824 2553 flags.go:64] FLAG: --topology-manager-scope="container" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632832 2553 flags.go:64] FLAG: --v="2" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632839 2553 flags.go:64] FLAG: --version="false" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632846 2553 flags.go:64] FLAG: --vmodule="" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632851 2553 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.632857 2553 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633033 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:12.927023 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633039 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633043 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633050 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633057 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633062 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633066 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633070 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633075 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633079 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633083 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633087 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633091 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633096 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633101 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633106 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633111 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633116 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633120 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633124 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:12.927809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633129 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633133 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633137 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633141 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633145 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633149 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633155 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633159 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633164 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633167 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633171 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633176 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633180 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633184 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633188 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633192 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633195 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633200 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633203 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633207 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:12.940077 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633211 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633215 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633219 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633222 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633226 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633230 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633235 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633239 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633245 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633251 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633255 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633259 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633264 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633271 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633276 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633280 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633284 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633287 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633291 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633298 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:12.940893 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633301 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633305 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633310 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633313 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633317 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633321 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633324 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633328 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633332 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633336 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633340 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633344 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633347 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633352 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633355 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633359 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633363 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633367 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633371 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633374 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:12.941691 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633378 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633381 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633385 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633389 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633393 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633398 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633402 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633406 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633409 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633413 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633417 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633421 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633426 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633430 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633435 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633438 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633442 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633446 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:12.942500 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.633449 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.634247 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.641263 2553 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.641276 2553 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.641294 2553 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.641300 2553 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641355 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641359 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641363 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641366 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641369 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641372 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641375 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641378 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641381 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:12.943293 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641384 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641386 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641389 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641392 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641394 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641397 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641399 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641402 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641405 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641407 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641410 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641412 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641415 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641418 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641421 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641424 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641427 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641433 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641435 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641438 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:12.944097 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641441 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641443 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641446 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641449 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641451 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641453 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641456 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641459 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641461 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641464 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641466 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641469 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641472 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641474 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641477 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641479 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641482 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641484 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641488 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641491 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:13.040540 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641494 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641497 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641499 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641501 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641504 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641507 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641510 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641512 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641515 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641519 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641522 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641524 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641528 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641531 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641534 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641536 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641539 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641542 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641546 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:13.041274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641549 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641552 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641555 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641557 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641560 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641563 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641565 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641568 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641570 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641572 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641575 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641577 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641580 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641582 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641585 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641587 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641590 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641592 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641595 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641598 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:13.042071 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641600 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641603 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641607 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641609 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641612 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641615 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641634 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641637 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641641 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641645 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641648 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.641653 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641765 2553 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641770 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 16 15:35:13.042888 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641773 2553 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641776 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641779 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641782 2553 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641785 2553 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641787 2553 feature_gate.go:328] unrecognized feature gate: Example2 Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641790 2553 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641792 2553 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641795 2553 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641797 2553 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641801 2553 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641805 2553 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641808 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641812 2553 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641815 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641818 2553 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641822 2553 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641825 2553 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641828 2553 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 16 15:35:13.043274 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641831 2553 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641833 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641837 2553 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641840 2553 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641844 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641846 2553 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641849 2553 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641852 2553 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641854 2553 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641856 2553 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641859 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641862 2553 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641864 2553 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641867 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641869 2553 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641872 2553 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641874 2553 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641877 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641879 2553 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641882 2553 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 16 15:35:13.044080 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641884 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641886 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641889 2553 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641891 2553 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641894 2553 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641896 2553 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641899 2553 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641902 2553 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641904 2553 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641907 2553 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641909 2553 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641912 2553 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641915 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641918 2553 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641921 2553 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641924 2553 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641930 2553 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641933 2553 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641936 2553 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 16 15:35:13.044895 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641939 2553 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641942 2553 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641944 2553 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641947 2553 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641950 2553 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641952 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641955 2553 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641958 2553 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641960 2553 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641963 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641965 2553 feature_gate.go:328] unrecognized feature gate: Example Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641968 2553 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641970 2553 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641973 2553 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641975 2553 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641977 2553 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641980 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641982 2553 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641985 2553 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641987 2553 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 16 15:35:13.046103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641990 2553 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641992 2553 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641995 2553 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641997 2553 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.641999 2553 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642002 2553 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642005 2553 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642009 2553 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642011 2553 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642015 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642017 2553 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642020 2553 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642022 2553 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642025 2553 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642027 2553 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642030 2553 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642032 2553 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642035 2553 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 16 15:35:13.046921 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:12.642037 2553 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.642043 2553 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.642131 2553 server.go:962] "Client rotation is on, will bootstrap in background" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.644481 2553 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.646207 2553 server.go:1019] "Starting client certificate rotation" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.646311 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.646354 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.649016 2553 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.654566 2553 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.664369 2553 log.go:25] "Validated CRI v1 runtime API" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.670756 2553 log.go:25] "Validated CRI v1 image API" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.672535 2553 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 16 15:35:13.047710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.675138 2553 fs.go:135] Filesystem UUIDs: map[5917dd9d-7649-4188-a306-a33563703042:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 b993e624-1940-4e01-8066-ee7a192a880c:/dev/nvme0n1p4] Jul 16 15:35:13.048104 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.675152 2553 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.680009 2553 manager.go:216] Machine: {Timestamp:2026-07-16 15:35:12.679544299 +0000 UTC m=+0.381096180 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3099372 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec22dfccdc9b9b73d8fd1320b6e2d371 SystemUUID:ec22dfcc-dc9b-9b73-d8fd-1320b6e2d371 BootID:0922598d-c1a8-4494-a5db-13cf642ead82 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:e9:b5:be:3d:d5 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:e9:b5:be:3d:d5 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:fa:2e:ef:16:d0:27 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.680103 2553 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.680168 2553 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.681482 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.683690 2553 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.683718 2553 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-168.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 16 15:35:13.048517 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.685343 2553 topology_manager.go:138] "Creating topology manager with none policy" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.685358 2553 container_manager_linux.go:309] "Creating device plugin manager" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.685369 2553 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.686179 2553 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.688149 2553 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.692901 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693031 2553 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693065 2553 kubelet.go:486] "Attempting to sync node with API server" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693077 2553 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693091 2553 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693105 2553 kubelet.go:398] "Adding apiserver pod source" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693120 2553 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693467 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.693485 2553 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.695768 2553 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698191 2553 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698217 2553 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698353 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698366 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698371 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698377 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698385 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.698390 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699035 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699041 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699047 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699052 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699068 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699078 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699123 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.699132 2553 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.702090 2553 server.go:1268] "Started kubelet" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.706552 2553 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.706641 2553 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.707372 2553 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.708159 2553 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.708927 2553 server.go:310] "Adding debug handlers to kubelet server" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.710560 2553 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-168.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.710584 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 16 15:35:13.048914 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.710849 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.713814 2553 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.716280 2553 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.716292 2553 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.716382 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717041 2553 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717049 2553 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717067 2553 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717053 2553 factory.go:55] Registering systemd factory Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717106 2553 factory.go:223] Registration of the systemd container factory successfully Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717209 2553 reconstruct.go:97] "Volume reconstruction finished" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717216 2553 reconciler.go:29] "Reconciler: start to sync state" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717318 2553 factory.go:153] Registering CRI-O factory Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717328 2553 factory.go:223] Registration of the crio container factory successfully Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717369 2553 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717389 2553 factory.go:103] Registering Raw factory Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717399 2553 manager.go:1196] Started watching for new ooms in manager Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.717694 2553 manager.go:349] Starting recovery of all containers Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.723760 2553 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.723674 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f0205b39 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.701922105 +0000 UTC m=+0.403473985,LastTimestamp:2026-07-16 15:35:12.701922105 +0000 UTC m=+0.403473985,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.727371 2553 manager.go:354] Recovery completed Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.729748 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tmpzr" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.729783 2553 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-168.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.737174 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-tmpzr" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.742129 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744086 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientMemory" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744108 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744118 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientPID" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744571 2553 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744580 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744580 2553 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.744648 2553 state_mem.go:36] "Initialized new in-memory state store" Jul 16 15:35:13.050131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.746953 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientMemory" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.746973 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.746984 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientPID" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.747073 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a3dd9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.744095131 +0000 UTC m=+0.445647011,LastTimestamp:2026-07-16 15:35:12.744095131 +0000 UTC m=+0.445647011,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.758312 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a420ab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.744112299 +0000 UTC m=+0.445664179,LastTimestamp:2026-07-16 15:35:12.744112299 +0000 UTC m=+0.445664179,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.771263 2553 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a44424 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.74412138 +0000 UTC m=+0.445673260,LastTimestamp:2026-07-16 15:35:12.74412138 +0000 UTC m=+0.445673260,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.054946 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.784325 2553 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-168.ec2.internal.18c2cf19f2a3dd9b\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a3dd9b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.744095131 +0000 UTC m=+0.445647011,LastTimestamp:2026-07-16 15:35:12.746965349 +0000 UTC m=+0.448517230,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.803778 2553 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-168.ec2.internal.18c2cf19f2a420ab\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a420ab default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.744112299 +0000 UTC m=+0.445664179,LastTimestamp:2026-07-16 15:35:12.746979327 +0000 UTC m=+0.448531207,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.816934 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.818216 2553 event.go:359] "Server rejected event (will not retry!)" err="events \"ip-10-0-134-168.ec2.internal.18c2cf19f2a44424\" is forbidden: User \"system:anonymous\" cannot patch resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-168.ec2.internal.18c2cf19f2a44424 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-168.ec2.internal,UID:ip-10-0-134-168.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-134-168.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-134-168.ec2.internal,},FirstTimestamp:2026-07-16 15:35:12.74412138 +0000 UTC m=+0.445673260,LastTimestamp:2026-07-16 15:35:12.746987878 +0000 UTC m=+0.448539759,Count:2,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-168.ec2.internal,}" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.880443 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.881917 2553 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.881960 2553 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.881979 2553 kubelet.go:2448] "Starting kubelet main sync loop" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.882009 2553 kubelet.go:2472] "Skipping pod synchronization" err="[container runtime status check may not have completed yet, PLEG is not healthy: pleg has yet to be successful]" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.886485 2553 policy_none.go:49] "None policy: Start" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.886499 2553 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.886511 2553 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.888764 2553 policy_none.go:47] "Start" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:12.891369 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.917504 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.937373 2553 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-134-168.ec2.internal\" not found" node="ip-10-0-134-168.ec2.internal" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:12.982482 2553 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 16 15:35:13.055355 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.017657 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.161497 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.118057 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.278307 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.183381 2553 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 16 15:35:13.278307 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.218790 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.388800 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.319180 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.501746 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.419574 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.522229 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.520077 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.628165 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.584426 2553 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 16 15:35:13.628165 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.598195 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:13.628165 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.620991 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.632138 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.631345 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:13.647351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.645733 2553 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 16 15:35:13.647351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.645836 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 15:35:13.647351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.645872 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 15:35:13.647351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.645883 2553 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 16 15:35:13.763129 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.693985 2553 apiserver.go:52] "Watching apiserver" Jul 16 15:35:13.763129 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.706118 2553 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 16 15:35:13.763129 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.720603 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:13.763129 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.721719 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.763129 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.738541 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.797577 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.821987 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.885233 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.885959 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-15 15:30:13 +0000 UTC" deadline="2028-02-11 03:53:25.154862934 +0000 UTC" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.885994 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13788h18m11.26887455s" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.886076 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientMemory" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.886126 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 15:35:13.890875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.886145 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientPID" Jul 16 15:35:14.027544 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:13.922213 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.027544 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:13.979405 2553 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:14.027544 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.022795 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.157894 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.078780 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.157894 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.078798 2553 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.157894 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.118238 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.157894 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.123457 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.157894 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.136279 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.271062 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.195286 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.271062 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.223734 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.390660 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.324347 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.390660 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.384730 2553 kubelet.go:2472] "Skipping pod synchronization" err="container runtime status check may not have completed yet" Jul 16 15:35:14.420309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420290 2553 manager.go:337] "Starting Device Plugin manager" Jul 16 15:35:14.420393 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.420323 2553 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 16 15:35:14.420393 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420333 2553 server.go:87] "Starting device plugin registration server" Jul 16 15:35:14.420619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420606 2553 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 16 15:35:14.420678 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420623 2553 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 16 15:35:14.420724 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420704 2553 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 16 15:35:14.420821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420799 2553 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 16 15:35:14.420821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.420813 2553 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 16 15:35:14.421392 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.421372 2553 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 16 15:35:14.421455 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.421430 2553 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.457540 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.457518 2553 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.457540 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.457537 2553 csi_plugin.go:399] Failed to initialize CSINode: error updating CSINode annotation: timed out waiting for the condition; caused by: nodes "ip-10-0-134-168.ec2.internal" not found Jul 16 15:35:14.521391 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.521361 2553 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 16 15:35:14.522404 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.522389 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientMemory" Jul 16 15:35:14.522481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.522417 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasNoDiskPressure" Jul 16 15:35:14.522481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.522428 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeHasSufficientPID" Jul 16 15:35:14.522481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.522453 2553 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-168.ec2.internal" Jul 16 15:35:14.532793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.532770 2553 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-168.ec2.internal" Jul 16 15:35:14.532856 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.532792 2553 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-168.ec2.internal\": node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.565934 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.565911 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.666329 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.666258 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.769250 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.769205 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.803595 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:14.803572 2553 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:14.872223 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.872191 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:14.975141 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:14.975048 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:15.078287 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:15.078242 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:15.180570 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:15.180528 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:15.283972 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:15.283877 2553 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-134-168.ec2.internal\" not found" Jul 16 15:35:15.351199 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.351174 2553 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:15.687220 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.687162 2553 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 16 15:35:15.719351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.719325 2553 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 16 15:35:15.731227 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.731209 2553 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 16 15:35:15.813839 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.813813 2553 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-rlhm7" Jul 16 15:35:15.821844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.821821 2553 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-rlhm7" Jul 16 15:35:15.988356 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.988238 2553 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal"] Jul 16 15:35:15.991142 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.991121 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:15.993349 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.993328 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=[] Jul 16 15:35:15.993349 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.993352 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-h59gl"] Jul 16 15:35:15.993470 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.993417 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:15.995926 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.995896 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h59gl" Jul 16 15:35:15.998388 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:15.998370 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/node-ca-4zz9k"] Jul 16 15:35:16.000811 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.000797 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-lqk7c"] Jul 16 15:35:16.000920 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.000904 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.002692 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.002676 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 16 15:35:16.002884 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.002868 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-kmfgh\"" Jul 16 15:35:16.003186 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.003172 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ovn-kubernetes/ovnkube-node-jzljd"] Jul 16 15:35:16.003291 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.003278 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.003366 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.003347 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:16.003470 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.003452 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.003514 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.003489 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.003998 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.003984 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 16 15:35:16.004075 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.004059 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.004212 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.004197 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.004258 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.004224 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 16 15:35:16.004787 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.004775 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-n2f7p\"" Jul 16 15:35:16.005934 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.005918 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp"] Jul 16 15:35:16.006051 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.006034 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.008588 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.008571 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-operator/iptables-alerter-tcz6r"] Jul 16 15:35:16.008777 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.008764 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.011025 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.011009 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-z96xk"] Jul 16 15:35:16.011141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.011126 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.013538 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.013523 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-r4749"] Jul 16 15:35:16.013674 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.013661 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.016405 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.016376 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 16 15:35:16.016651 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.016634 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 16 15:35:16.017118 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017087 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 16 15:35:16.017336 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017320 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.017409 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017320 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.017409 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017373 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.017567 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017375 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-vkbkj\"" Jul 16 15:35:16.017812 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017794 2553 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 16 15:35:16.017882 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.017820 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-gl22h\"" Jul 16 15:35:16.018112 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.018091 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/konnectivity-agent-gbvcg"] Jul 16 15:35:16.018208 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.018195 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.018977 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.018961 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.019237 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019222 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.019464 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019446 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.019600 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019583 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.019700 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019615 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 16 15:35:16.019700 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019689 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m2gxb\"" Jul 16 15:35:16.019839 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019737 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-jbnh9\"" Jul 16 15:35:16.019899 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019853 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.019899 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019861 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 16 15:35:16.019987 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.019973 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 16 15:35:16.020047 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.020031 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.020516 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.020498 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 16 15:35:16.020858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.020842 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-48sll"] Jul 16 15:35:16.020938 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.020892 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-8v9z6\"" Jul 16 15:35:16.020997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.020962 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.021355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.021332 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 16 15:35:16.023325 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.023305 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 16 15:35:16.023405 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.023390 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gtpqd"] Jul 16 15:35:16.023463 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.023455 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:16.023543 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.023524 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:16.025617 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.025601 2553 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.025928 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.025912 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-node-tuning-operator/tuned-xwv7t"] Jul 16 15:35:16.026026 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.026013 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.026090 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.026068 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:16.027375 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.027356 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-scbd2\"" Jul 16 15:35:16.027437 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.027361 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 16 15:35:16.028485 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.028467 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.030920 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.030904 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ld2nd\"" Jul 16 15:35:16.031358 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.031344 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:16.031584 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.031569 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:16.037239 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037225 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal"] Jul 16 15:35:16.037332 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037315 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-multus\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037381 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-multus-certs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037381 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037361 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-slash\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037381 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037378 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-ovn\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037529 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037395 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037529 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037451 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-kubelet\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037529 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037467 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 15:35:16.037529 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037485 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-conf-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037529 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037517 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-etc-kubernetes\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037542 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-node-log\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037606 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-device-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037647 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k6784\" (UniqueName: \"kubernetes.io/projected/da228326-59ec-470d-b3e9-d9634725a7b0-kube-api-access-k6784\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037674 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037696 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-daemon-config\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037710 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8338837b-8db4-4083-bab5-0a846db22992-serviceca\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037730 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-var-lib-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037745 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-log-socket\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037779 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jpqfq\" (UniqueName: \"kubernetes.io/projected/ca808dec-8a96-4654-878e-734591ee12ed-kube-api-access-jpqfq\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.037797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037801 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-netns\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-kubelet\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84ea9286-2f1d-4d42-848e-e48204a86073-hosts-file\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037896 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-env-overrides\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037921 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-system-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037946 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-cnibin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037974 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l8qqk\" (UniqueName: \"kubernetes.io/projected/8338837b-8db4-4083-bab5-0a846db22992-kube-api-access-l8qqk\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.037999 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-systemd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038026 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038056 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ea9286-2f1d-4d42-848e-e48204a86073-tmp-dir\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038083 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-netns\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038107 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-script-lib\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038133 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da228326-59ec-470d-b3e9-d9634725a7b0-iptables-alerter-script\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038160 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lzd7x\" (UniqueName: \"kubernetes.io/projected/87f476dc-4dbe-4b63-ba23-4e6177babe08-kube-api-access-lzd7x\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.038295 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038185 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038217 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-netd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038237 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038255 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-config\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038281 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038310 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038338 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da228326-59ec-470d-b3e9-d9634725a7b0-host-slash\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038364 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjfs\" (UniqueName: \"kubernetes.io/projected/b064e026-71d9-40c0-a0ec-3f76c79562eb-kube-api-access-4cjfs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038380 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-os-release\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038401 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-systemd-units\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-cni-binary-copy\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038435 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-bin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038463 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2zmp7\" (UniqueName: \"kubernetes.io/projected/84ea9286-2f1d-4d42-848e-e48204a86073-kube-api-access-2zmp7\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038487 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-socket-dir-parent\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.038765 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038517 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r27sx\" (UniqueName: \"kubernetes.io/projected/6ffde159-037f-4b6e-ab53-58f0e26ecae4-kube-api-access-r27sx\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038539 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038555 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038570 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f8e6baf277b94b5597c0f3301bb4ca1d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-168.ec2.internal\" (UID: \"f8e6baf277b94b5597c0f3301bb4ca1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038584 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8338837b-8db4-4083-bab5-0a846db22992-host\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038599 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-etc-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038621 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-bin\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038663 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038683 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-k8s-cni-cncf-io\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038698 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-hostroot\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.039235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.038714 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.042902 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.042887 2553 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 16 15:35:16.043088 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.043073 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal"] Jul 16 15:35:16.139360 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139336 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-conf-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139366 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-etc-kubernetes\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139389 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-system-cni-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.139452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139414 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.139452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139440 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-node-log\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.139452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139440 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-conf-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139473 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-etc-kubernetes\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139490 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-device-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139498 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-node-log\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139513 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139538 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k6784\" (UniqueName: \"kubernetes.io/projected/da228326-59ec-470d-b3e9-d9634725a7b0-kube-api-access-k6784\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139552 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-device-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139564 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139586 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-daemon-config\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139619 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139615 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139664 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-systemd\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139674 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139691 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8338837b-8db4-4083-bab5-0a846db22992-serviceca\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139714 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-var-lib-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139730 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-log-socket\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139747 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-jpqfq\" (UniqueName: \"kubernetes.io/projected/ca808dec-8a96-4654-878e-734591ee12ed-kube-api-access-jpqfq\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139765 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-netns\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139797 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-kubelet\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139815 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-netns\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139815 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-log-socket\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-kubelet-config\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139841 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-kubelet\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139853 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-tmp\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139871 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84ea9286-2f1d-4d42-848e-e48204a86073-hosts-file\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.139997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139886 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-env-overrides\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139912 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-system-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139949 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/84ea9286-2f1d-4d42-848e-e48204a86073-hosts-file\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139960 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-system-cni-dir\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.139979 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-cnibin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140000 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l8qqk\" (UniqueName: \"kubernetes.io/projected/8338837b-8db4-4083-bab5-0a846db22992-kube-api-access-l8qqk\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140024 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-var-lib-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140028 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-systemd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140071 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140079 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-systemd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140088 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-cnibin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140090 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtm9q\" (UniqueName: \"kubernetes.io/projected/bff48646-a42e-4216-97c5-d776060c9b08-kube-api-access-xtm9q\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140119 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140135 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ea9286-2f1d-4d42-848e-e48204a86073-tmp-dir\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140158 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:16.140677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140175 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140195 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140228 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-netns\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-script-lib\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140272 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-run-netns\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140352 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/84ea9286-2f1d-4d42-848e-e48204a86073-tmp-dir\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140368 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140407 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e5d41d-c3d4-423c-88a5-c15a648843dc-konnectivity-ca\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140437 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140465 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-sys\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140501 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-etc-tuned\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140533 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da228326-59ec-470d-b3e9-d9634725a7b0-iptables-alerter-script\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140561 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e5d41d-c3d4-423c-88a5-c15a648843dc-agent-certs\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140589 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-lzd7x\" (UniqueName: \"kubernetes.io/projected/87f476dc-4dbe-4b63-ba23-4e6177babe08-kube-api-access-lzd7x\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.141372 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140618 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140659 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-netd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140685 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140711 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-config\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140717 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140735 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140757 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140760 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140784 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/8338837b-8db4-4083-bab5-0a846db22992-serviceca\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da228326-59ec-470d-b3e9-d9634725a7b0-host-slash\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140811 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjfs\" (UniqueName: \"kubernetes.io/projected/b064e026-71d9-40c0-a0ec-3f76c79562eb-kube-api-access-4cjfs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140818 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-sys-fs\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140826 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-netd\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140830 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-os-release\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140853 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da228326-59ec-470d-b3e9-d9634725a7b0-host-slash\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140881 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-registration-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140897 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-systemd-units\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140922 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-cni-binary-copy\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140939 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-bin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140955 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2zmp7\" (UniqueName: \"kubernetes.io/projected/84ea9286-2f1d-4d42-848e-e48204a86073-kube-api-access-2zmp7\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.140978 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-cnibin\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141004 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-os-release\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141006 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141036 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-bin\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141067 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-socket-dir-parent\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141079 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-systemd-units\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141094 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141154 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-dbus\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.142554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141174 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/da228326-59ec-470d-b3e9-d9634725a7b0-iptables-alerter-script\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141183 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-var-lib-kubelet\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141154 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-socket-dir-parent\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141208 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-host\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.141227 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141235 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r27sx\" (UniqueName: \"kubernetes.io/projected/6ffde159-037f-4b6e-ab53-58f0e26ecae4-kube-api-access-r27sx\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.141329 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:16.64130606 +0000 UTC m=+4.342857949 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141459 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-cni-binary-copy\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141459 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/b064e026-71d9-40c0-a0ec-3f76c79562eb-multus-daemon-config\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141483 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141524 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5cnkg\" (UniqueName: \"kubernetes.io/projected/aacaf418-8d1b-4748-af53-3c57864d3ddf-kube-api-access-5cnkg\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141534 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-socket-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141537 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-etc-selinux\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.142994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141551 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysconfig\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141537 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-env-overrides\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141586 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-kubernetes\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141605 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f8e6baf277b94b5597c0f3301bb4ca1d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-168.ec2.internal\" (UID: \"f8e6baf277b94b5597c0f3301bb4ca1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141628 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8338837b-8db4-4083-bab5-0a846db22992-host\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141659 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/f8e6baf277b94b5597c0f3301bb4ca1d-config\") pod \"kube-apiserver-proxy-ip-10-0-134-168.ec2.internal\" (UID: \"f8e6baf277b94b5597c0f3301bb4ca1d\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-etc-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141694 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-bin\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141709 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/8338837b-8db4-4083-bab5-0a846db22992-host\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141718 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-etc-openvswitch\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141765 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-k8s-cni-cncf-io\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141771 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-cni-bin\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141787 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-hostroot\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141805 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-k8s-cni-cncf-io\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141812 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-conf\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141839 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141841 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/ca808dec-8a96-4654-878e-734591ee12ed-kubelet-dir\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141845 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-hostroot\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141864 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-multus\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141890 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-multus-certs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141898 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/fdf2ddabf14f6f15b51026dc5c8b42a8-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal\" (UID: \"fdf2ddabf14f6f15b51026dc5c8b42a8\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141910 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-var-lib-cni-multus\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141917 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-lib-modules\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141942 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/b064e026-71d9-40c0-a0ec-3f76c79562eb-host-run-multus-certs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141944 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-slash\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141982 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-script-lib\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.141984 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-ovn\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142023 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-run-ovn\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.143891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142028 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovnkube-config\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142032 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142097 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-os-release\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142126 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-run\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142156 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-kubelet\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142184 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-modprobe-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142158 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-slash\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142213 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/6ffde159-037f-4b6e-ab53-58f0e26ecae4-host-kubelet\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.144420 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.142296 2553 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 16 15:35:16.145646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.145625 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/6ffde159-037f-4b6e-ab53-58f0e26ecae4-ovn-node-metrics-cert\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.153386 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.153359 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k6784\" (UniqueName: \"kubernetes.io/projected/da228326-59ec-470d-b3e9-d9634725a7b0-kube-api-access-k6784\") pod \"iptables-alerter-tcz6r\" (UID: \"da228326-59ec-470d-b3e9-d9634725a7b0\") " pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.154672 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.154649 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-jpqfq\" (UniqueName: \"kubernetes.io/projected/ca808dec-8a96-4654-878e-734591ee12ed-kube-api-access-jpqfq\") pod \"aws-ebs-csi-driver-node-v4hmp\" (UID: \"ca808dec-8a96-4654-878e-734591ee12ed\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.154895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.154855 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjfs\" (UniqueName: \"kubernetes.io/projected/b064e026-71d9-40c0-a0ec-3f76c79562eb-kube-api-access-4cjfs\") pod \"multus-h59gl\" (UID: \"b064e026-71d9-40c0-a0ec-3f76c79562eb\") " pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.155079 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.155060 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l8qqk\" (UniqueName: \"kubernetes.io/projected/8338837b-8db4-4083-bab5-0a846db22992-kube-api-access-l8qqk\") pod \"node-ca-4zz9k\" (UID: \"8338837b-8db4-4083-bab5-0a846db22992\") " pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.155137 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.155072 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r27sx\" (UniqueName: \"kubernetes.io/projected/6ffde159-037f-4b6e-ab53-58f0e26ecae4-kube-api-access-r27sx\") pod \"ovnkube-node-jzljd\" (UID: \"6ffde159-037f-4b6e-ab53-58f0e26ecae4\") " pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.155369 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.155352 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-lzd7x\" (UniqueName: \"kubernetes.io/projected/87f476dc-4dbe-4b63-ba23-4e6177babe08-kube-api-access-lzd7x\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.155590 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.155575 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2zmp7\" (UniqueName: \"kubernetes.io/projected/84ea9286-2f1d-4d42-848e-e48204a86073-kube-api-access-2zmp7\") pod \"node-resolver-z96xk\" (UID: \"84ea9286-2f1d-4d42-848e-e48204a86073\") " pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.242664 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242560 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-cnibin\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.242664 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242615 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242708 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-cnibin\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.242718 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-dbus\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242788 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-var-lib-kubelet\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242812 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-host\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.242844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242841 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5cnkg\" (UniqueName: \"kubernetes.io/projected/aacaf418-8d1b-4748-af53-3c57864d3ddf-kube-api-access-5cnkg\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242863 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysconfig\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242887 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-kubernetes\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242883 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-var-lib-kubelet\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.242906 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:16.742882986 +0000 UTC m=+4.444434871 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242925 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-host\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242952 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-conf\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242962 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-dbus\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242974 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-kubernetes\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.242982 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-lib-modules\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243030 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-os-release\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243057 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-run\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243077 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-conf\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243082 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-modprobe-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243111 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-system-cni-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243118 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-run\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243140 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-os-release\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243158 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-system-cni-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243149 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-lib-modules\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243163 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243191 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysconfig\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243201 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-systemd\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243202 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-modprobe-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243234 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-kubelet-config\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243257 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-tmp\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243279 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtm9q\" (UniqueName: \"kubernetes.io/projected/bff48646-a42e-4216-97c5-d776060c9b08-kube-api-access-xtm9q\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243279 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-systemd\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243298 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:16.243821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243299 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/aacaf418-8d1b-4748-af53-3c57864d3ddf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243315 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243355 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/116501b0-acde-4960-b464-465ccd1d27a6-kubelet-config\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243446 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243472 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e5d41d-c3d4-423c-88a5-c15a648843dc-konnectivity-ca\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243500 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243601 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-etc-sysctl-d\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243632 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-sys\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243685 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-etc-tuned\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243710 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e5d41d-c3d4-423c-88a5-c15a648843dc-agent-certs\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243775 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/bff48646-a42e-4216-97c5-d776060c9b08-sys\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.243877 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-binary-copy\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.244107 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e3e5d41d-c3d4-423c-88a5-c15a648843dc-konnectivity-ca\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.244339 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.244122 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.245053 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.244382 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/aacaf418-8d1b-4748-af53-3c57864d3ddf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.245645 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.245611 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-tmp\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.246078 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.246059 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e3e5d41d-c3d4-423c-88a5-c15a648843dc-agent-certs\") pod \"konnectivity-agent-gbvcg\" (UID: \"e3e5d41d-c3d4-423c-88a5-c15a648843dc\") " pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.246628 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.246612 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/bff48646-a42e-4216-97c5-d776060c9b08-etc-tuned\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.255000 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.254982 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:16.255000 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.255002 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:16.255162 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.255012 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:16.255162 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.255066 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:16.755053573 +0000 UTC m=+4.456605468 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:16.256632 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.256604 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtm9q\" (UniqueName: \"kubernetes.io/projected/bff48646-a42e-4216-97c5-d776060c9b08-kube-api-access-xtm9q\") pod \"tuned-xwv7t\" (UID: \"bff48646-a42e-4216-97c5-d776060c9b08\") " pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.257307 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.257288 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-5cnkg\" (UniqueName: \"kubernetes.io/projected/aacaf418-8d1b-4748-af53-3c57864d3ddf-kube-api-access-5cnkg\") pod \"multus-additional-cni-plugins-r4749\" (UID: \"aacaf418-8d1b-4748-af53-3c57864d3ddf\") " pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.339809 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.339773 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.346240 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.345766 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" Jul 16 15:35:16.359872 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.359851 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h59gl" Jul 16 15:35:16.379785 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.379764 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-4zz9k" Jul 16 15:35:16.384752 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.384731 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:16.394762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.394746 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" Jul 16 15:35:16.414451 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.414432 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-tcz6r" Jul 16 15:35:16.430991 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.430970 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-z96xk" Jul 16 15:35:16.444973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.444951 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-r4749" Jul 16 15:35:16.468004 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.467976 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:16.475869 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.475851 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" Jul 16 15:35:16.647395 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.647361 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:16.647551 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.647520 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:16.647589 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.647583 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:17.647565922 +0000 UTC m=+5.349117795 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:16.748785 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.748745 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:16.749347 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.748900 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:16.749347 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.748967 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:17.748951134 +0000 UTC m=+5.450503006 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:16.825419 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.825365 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 15:30:16 +0000 UTC" deadline="2027-12-11 14:04:38.66855943 +0000 UTC" Jul 16 15:35:16.825419 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.825412 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12310h29m21.843149519s" Jul 16 15:35:16.849870 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.849840 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:16.850026 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.850006 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:16.850068 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.850032 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:16.850068 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.850042 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:16.850133 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:16.850098 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:17.850083463 +0000 UTC m=+5.551635336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:16.967224 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:16.967185 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb064e026_71d9_40c0_a0ec_3f76c79562eb.slice/crio-da9e6aa970f7f7bd1d7f5033b5ff3af21b039f56aee222d3e5b831670b616369 WatchSource:0}: Error finding container da9e6aa970f7f7bd1d7f5033b5ff3af21b039f56aee222d3e5b831670b616369: Status 404 returned error can't find the container with id da9e6aa970f7f7bd1d7f5033b5ff3af21b039f56aee222d3e5b831670b616369 Jul 16 15:35:16.967861 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:16.967780 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaacaf418_8d1b_4748_af53_3c57864d3ddf.slice/crio-2a7a99015fc16c2a6316627a3fdb32d5c634cb102b75300200c87e41c2161eb1 WatchSource:0}: Error finding container 2a7a99015fc16c2a6316627a3fdb32d5c634cb102b75300200c87e41c2161eb1: Status 404 returned error can't find the container with id 2a7a99015fc16c2a6316627a3fdb32d5c634cb102b75300200c87e41c2161eb1 Jul 16 15:35:16.971495 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:16.971465 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8e6baf277b94b5597c0f3301bb4ca1d.slice/crio-236771a39308b85e099aff06729027fcaf2533c493e592b371fcb7c39c402faf WatchSource:0}: Error finding container 236771a39308b85e099aff06729027fcaf2533c493e592b371fcb7c39c402faf: Status 404 returned error can't find the container with id 236771a39308b85e099aff06729027fcaf2533c493e592b371fcb7c39c402faf Jul 16 15:35:16.972889 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:16.972840 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podca808dec_8a96_4654_878e_734591ee12ed.slice/crio-f9b4e5e4489986cb5be7393848401501bd45b29417345826a2998f968168544d WatchSource:0}: Error finding container f9b4e5e4489986cb5be7393848401501bd45b29417345826a2998f968168544d: Status 404 returned error can't find the container with id f9b4e5e4489986cb5be7393848401501bd45b29417345826a2998f968168544d Jul 16 15:35:16.972989 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:16.972931 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 15:35:16.975048 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:16.975028 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3e5d41d_c3d4_423c_88a5_c15a648843dc.slice/crio-c5618e026ab052f872e2d1a728f9405c0bbe74f114db94d8146960f409c11599 WatchSource:0}: Error finding container c5618e026ab052f872e2d1a728f9405c0bbe74f114db94d8146960f409c11599: Status 404 returned error can't find the container with id c5618e026ab052f872e2d1a728f9405c0bbe74f114db94d8146960f409c11599 Jul 16 15:35:17.011340 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.011317 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6ffde159_037f_4b6e_ab53_58f0e26ecae4.slice/crio-4df9ad80d6ddd1d6ad01cc3a9559b1cc2c34d5eeb32c7f77b1574bf7d3e24dc5 WatchSource:0}: Error finding container 4df9ad80d6ddd1d6ad01cc3a9559b1cc2c34d5eeb32c7f77b1574bf7d3e24dc5: Status 404 returned error can't find the container with id 4df9ad80d6ddd1d6ad01cc3a9559b1cc2c34d5eeb32c7f77b1574bf7d3e24dc5 Jul 16 15:35:17.012506 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.012487 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdf2ddabf14f6f15b51026dc5c8b42a8.slice/crio-627440a063b51336db17483dd2d4fd2714306c2b19aca484ec62a1df3e59601e WatchSource:0}: Error finding container 627440a063b51336db17483dd2d4fd2714306c2b19aca484ec62a1df3e59601e: Status 404 returned error can't find the container with id 627440a063b51336db17483dd2d4fd2714306c2b19aca484ec62a1df3e59601e Jul 16 15:35:17.076812 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.076710 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda228326_59ec_470d_b3e9_d9634725a7b0.slice/crio-e6f8f19367648f911d529316a30db0b7a2579cf4828f86641f60f424fbc4f06e WatchSource:0}: Error finding container e6f8f19367648f911d529316a30db0b7a2579cf4828f86641f60f424fbc4f06e: Status 404 returned error can't find the container with id e6f8f19367648f911d529316a30db0b7a2579cf4828f86641f60f424fbc4f06e Jul 16 15:35:17.087867 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.087843 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8338837b_8db4_4083_bab5_0a846db22992.slice/crio-a7102b85a35a33f3b768255fc625af7689fca6df368f672c24fc3362d0ad4e64 WatchSource:0}: Error finding container a7102b85a35a33f3b768255fc625af7689fca6df368f672c24fc3362d0ad4e64: Status 404 returned error can't find the container with id a7102b85a35a33f3b768255fc625af7689fca6df368f672c24fc3362d0ad4e64 Jul 16 15:35:17.586434 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.586394 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbff48646_a42e_4216_97c5_d776060c9b08.slice/crio-be8f2d9f10f160d8d77802c56be21a6ce6961effee875805b1553311ea48d656 WatchSource:0}: Error finding container be8f2d9f10f160d8d77802c56be21a6ce6961effee875805b1553311ea48d656: Status 404 returned error can't find the container with id be8f2d9f10f160d8d77802c56be21a6ce6961effee875805b1553311ea48d656 Jul 16 15:35:17.657362 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.656753 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:17.657362 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.656936 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:17.657362 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.657004 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:19.656984036 +0000 UTC m=+7.358535921 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:17.757858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.757815 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:17.758717 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.758036 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:17.758717 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.758104 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:19.758083792 +0000 UTC m=+7.459635669 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:17.829894 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.829787 2553 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-15 15:30:16 +0000 UTC" deadline="2027-12-15 13:10:44.27699214 +0000 UTC" Jul 16 15:35:17.829894 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.829857 2553 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12405h35m26.447140083s" Jul 16 15:35:17.831022 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:17.830990 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84ea9286_2f1d_4d42_848e_e48204a86073.slice/crio-d7cab24825c292ed3c031f022950284ef7bd9a0d65339b57bf4d31d60edfff52 WatchSource:0}: Error finding container d7cab24825c292ed3c031f022950284ef7bd9a0d65339b57bf4d31d60edfff52: Status 404 returned error can't find the container with id d7cab24825c292ed3c031f022950284ef7bd9a0d65339b57bf4d31d60edfff52 Jul 16 15:35:17.859095 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.859065 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:17.859278 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.859257 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:17.859349 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.859287 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:17.859349 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.859301 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:17.859503 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.859361 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:19.85934232 +0000 UTC m=+7.560894199 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:17.882844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.882812 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:17.883004 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.882951 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:17.883434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.883412 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:17.883556 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.883522 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:17.883632 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.883608 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:17.883762 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:17.883741 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:17.903861 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.903768 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z96xk" event={"ID":"84ea9286-2f1d-4d42-848e-e48204a86073","Type":"ContainerStarted","Data":"d7cab24825c292ed3c031f022950284ef7bd9a0d65339b57bf4d31d60edfff52"} Jul 16 15:35:17.913484 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.913448 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4zz9k" event={"ID":"8338837b-8db4-4083-bab5-0a846db22992","Type":"ContainerStarted","Data":"a7102b85a35a33f3b768255fc625af7689fca6df368f672c24fc3362d0ad4e64"} Jul 16 15:35:17.917597 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.917567 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"4df9ad80d6ddd1d6ad01cc3a9559b1cc2c34d5eeb32c7f77b1574bf7d3e24dc5"} Jul 16 15:35:17.925998 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.925902 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gbvcg" event={"ID":"e3e5d41d-c3d4-423c-88a5-c15a648843dc","Type":"ContainerStarted","Data":"c5618e026ab052f872e2d1a728f9405c0bbe74f114db94d8146960f409c11599"} Jul 16 15:35:17.935931 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.935902 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" event={"ID":"f8e6baf277b94b5597c0f3301bb4ca1d","Type":"ContainerStarted","Data":"236771a39308b85e099aff06729027fcaf2533c493e592b371fcb7c39c402faf"} Jul 16 15:35:17.940400 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.940362 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerStarted","Data":"2a7a99015fc16c2a6316627a3fdb32d5c634cb102b75300200c87e41c2161eb1"} Jul 16 15:35:17.948119 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.948090 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h59gl" event={"ID":"b064e026-71d9-40c0-a0ec-3f76c79562eb","Type":"ContainerStarted","Data":"da9e6aa970f7f7bd1d7f5033b5ff3af21b039f56aee222d3e5b831670b616369"} Jul 16 15:35:17.952065 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.952023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" event={"ID":"bff48646-a42e-4216-97c5-d776060c9b08","Type":"ContainerStarted","Data":"be8f2d9f10f160d8d77802c56be21a6ce6961effee875805b1553311ea48d656"} Jul 16 15:35:17.966294 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.965393 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tcz6r" event={"ID":"da228326-59ec-470d-b3e9-d9634725a7b0","Type":"ContainerStarted","Data":"e6f8f19367648f911d529316a30db0b7a2579cf4828f86641f60f424fbc4f06e"} Jul 16 15:35:17.970740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.967639 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" event={"ID":"fdf2ddabf14f6f15b51026dc5c8b42a8","Type":"ContainerStarted","Data":"627440a063b51336db17483dd2d4fd2714306c2b19aca484ec62a1df3e59601e"} Jul 16 15:35:17.973827 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:17.973798 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" event={"ID":"ca808dec-8a96-4654-878e-734591ee12ed","Type":"ContainerStarted","Data":"f9b4e5e4489986cb5be7393848401501bd45b29417345826a2998f968168544d"} Jul 16 15:35:19.674456 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.674413 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:19.675029 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.674646 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:19.675029 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.674738 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:23.674717295 +0000 UTC m=+11.376269186 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:19.775922 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.775879 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:19.776111 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.776049 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:19.776177 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.776115 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:23.776097472 +0000 UTC m=+11.477649356 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:19.876905 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.876859 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:19.877095 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.877040 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:19.877095 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.877071 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:19.877095 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.877086 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:19.877335 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.877150 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:23.877129639 +0000 UTC m=+11.578681510 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:19.882500 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.882218 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:19.882500 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.882344 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:19.882500 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.882409 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:19.882500 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.882492 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:19.882855 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:19.882535 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:19.882855 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:19.882618 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:21.882956 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:21.883123 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:21.883529 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:21.883649 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:21.883723 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:21.883851 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:21.883801 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:23.712945 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.712900 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:23.713439 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.713030 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:23.713439 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.713110 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:31.713086426 +0000 UTC m=+19.414638394 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:23.814278 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.814237 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:23.814482 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.814413 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:23.814549 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.814511 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:31.814487196 +0000 UTC m=+19.516039076 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:23.883233 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.883145 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:23.883233 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.883185 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:23.883233 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.883185 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:23.883540 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.883303 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:23.883540 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.883392 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:23.883540 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.883483 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:23.915296 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:23.915259 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:23.915480 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.915398 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:23.915480 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.915417 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:23.915480 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.915427 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:23.915632 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:23.915488 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:31.915467889 +0000 UTC m=+19.617019770 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:25.882831 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:25.882794 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:25.883327 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:25.882935 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:25.883327 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:25.883291 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:25.883437 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:25.883374 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:25.883437 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:25.883422 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:25.883534 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:25.883499 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:27.883103 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:27.883067 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:27.883103 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:27.883099 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:27.883706 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:27.883097 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:27.883706 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:27.883198 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:27.883706 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:27.883313 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:27.883706 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:27.883428 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:29.883080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:29.883041 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:29.883080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:29.883079 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:29.883550 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:29.883060 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:29.883550 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:29.883158 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:29.883550 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:29.883254 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:29.883550 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:29.883346 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:31.771441 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.771407 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:31.771925 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.771535 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:31.771925 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.771603 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:47.771582047 +0000 UTC m=+35.473133915 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:31.872715 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.872679 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:31.872876 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.872814 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:31.872932 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.872890 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:47.872871103 +0000 UTC m=+35.574422971 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:31.882655 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.882630 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:31.882786 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.882634 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:31.882847 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.882801 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:31.882847 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.882722 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:31.882847 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.882635 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:31.882994 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.882949 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:31.973188 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:31.973161 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:31.973346 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.973329 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:31.973452 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.973352 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:31.973452 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.973366 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:31.973452 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:31.973425 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:47.973408197 +0000 UTC m=+35.674960073 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:33.882341 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:33.882309 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:33.882831 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:33.882428 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:33.882831 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:33.882441 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:33.882831 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:33.882439 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:33.882831 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:33.882545 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:33.882831 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:33.882652 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:35.882857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:35.882821 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:35.882857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:35.882847 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:35.883330 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:35.882866 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:35.883330 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:35.882966 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:35.883330 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:35.883050 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:35.883330 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:35.883137 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:37.883211 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:37.883181 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:37.883700 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:37.883212 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:37.883700 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:37.883217 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:37.883700 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:37.883282 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:37.883700 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:37.883389 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:37.883700 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:37.883486 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:38.032151 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.032129 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:35:38.032586 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.032564 2553 generic.go:356] "Generic (PLEG): container finished" podID="6ffde159-037f-4b6e-ab53-58f0e26ecae4" containerID="2ca5660e48162c26ea84f165896474f57e00455a40091edb2b92bb599fa62e79" exitCode=1 Jul 16 15:35:38.032703 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.032652 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerDied","Data":"2ca5660e48162c26ea84f165896474f57e00455a40091edb2b92bb599fa62e79"} Jul 16 15:35:38.032762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.032738 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"ae39587d30b09511189b95b882b3c8212fa6e3a10560c9328f0f2d3f0c6e5026"} Jul 16 15:35:38.038665 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.038577 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" event={"ID":"f8e6baf277b94b5597c0f3301bb4ca1d","Type":"ContainerStarted","Data":"e9b1deb5178f1fe9eac7cbe2388ebbabc668149ce3673ed50b2294c5d61e5a04"} Jul 16 15:35:38.044234 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.044206 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h59gl" event={"ID":"b064e026-71d9-40c0-a0ec-3f76c79562eb","Type":"ContainerStarted","Data":"3a97daf94dbcdbc5e2cba15e7616685968c1987370c1ae875d3585f7d49d1fd7"} Jul 16 15:35:38.045842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.045810 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" event={"ID":"bff48646-a42e-4216-97c5-d776060c9b08","Type":"ContainerStarted","Data":"4f703f13cb6a0de932d3fdbc7d424e07779959b22467266d7d9fbb19cdaec673"} Jul 16 15:35:38.065105 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.065041 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-168.ec2.internal" podStartSLOduration=22.065026721 podStartE2EDuration="22.065026721s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:38.064882195 +0000 UTC m=+25.766434087" watchObservedRunningTime="2026-07-16 15:35:38.065026721 +0000 UTC m=+25.766578610" Jul 16 15:35:38.094573 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:38.094535 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xwv7t" podStartSLOduration=3.045920923 podStartE2EDuration="23.094520712s" totalImagesPullingTime="20.048599789s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:38.093724043 +0000 UTC m=+25.795275932" watchObservedRunningTime="2026-07-16 15:35:38.094520712 +0000 UTC m=+25.796072602" Jul 16 15:35:39.050008 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.049592 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-z96xk" event={"ID":"84ea9286-2f1d-4d42-848e-e48204a86073","Type":"ContainerStarted","Data":"846bba6973303d4e4e87f7428a25240b2f1e7eb3559713e3310a3f76b591a3e8"} Jul 16 15:35:39.051521 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.051372 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-4zz9k" event={"ID":"8338837b-8db4-4083-bab5-0a846db22992","Type":"ContainerStarted","Data":"591f99f66a918069cb2c2e6a0c43006e13143b7356fe68bed9ef61a7aa5b5bce"} Jul 16 15:35:39.054637 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.054605 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:35:39.055056 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.055037 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"7ed9bd0b0aab4159bf614ac51f89b4865a42724e36695b6e65d362d908290dd4"} Jul 16 15:35:39.055124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.055112 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"36bd9596b2880ba163bd0be19a0db69e17d9ab97087bdc19561a09059e448443"} Jul 16 15:35:39.055179 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.055161 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"d40eb8898ac50bc0b717b3a95faf5a1f232152f56b7aa8f1e437d73313948a42"} Jul 16 15:35:39.055250 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.055236 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"4dc2907c50db40c2778d41d9e925a66b3db589b52cd79afc7aca543cd485c362"} Jul 16 15:35:39.056537 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.056509 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-gbvcg" event={"ID":"e3e5d41d-c3d4-423c-88a5-c15a648843dc","Type":"ContainerStarted","Data":"2724e82177b30cf105873c14fe791269d2c6589c4212ed7d81fc1a4e255a38e6"} Jul 16 15:35:39.058036 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.058015 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="cc99cbbee45e52c6044c986148b6c1c3f2568b205ee8ac3f21b7a5a90a7d3903" exitCode=0 Jul 16 15:35:39.058131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.058090 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"cc99cbbee45e52c6044c986148b6c1c3f2568b205ee8ac3f21b7a5a90a7d3903"} Jul 16 15:35:39.061309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.061287 2553 generic.go:356] "Generic (PLEG): container finished" podID="fdf2ddabf14f6f15b51026dc5c8b42a8" containerID="172396535bc06096ce4d97e53aa0d3165e74c0c19d35969174508dc24e883612" exitCode=0 Jul 16 15:35:39.061411 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.061387 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" event={"ID":"fdf2ddabf14f6f15b51026dc5c8b42a8","Type":"ContainerDied","Data":"172396535bc06096ce4d97e53aa0d3165e74c0c19d35969174508dc24e883612"} Jul 16 15:35:39.063249 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.063195 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" event={"ID":"ca808dec-8a96-4654-878e-734591ee12ed","Type":"ContainerStarted","Data":"397b420fa6f4c7d769bc3f3abd0c79e4e733d8d3c6f20fee6d8cc42c4837641a"} Jul 16 15:35:39.094098 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.093216 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-h59gl" podStartSLOduration=3.36133924 podStartE2EDuration="24.093191103s" totalImagesPullingTime="20.731851863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:38.149081765 +0000 UTC m=+25.850633654" watchObservedRunningTime="2026-07-16 15:35:39.093191103 +0000 UTC m=+26.794742994" Jul 16 15:35:39.094098 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.093856 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-z96xk" podStartSLOduration=4.287769867 podStartE2EDuration="24.093845787s" totalImagesPullingTime="19.80607592s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:39.092793774 +0000 UTC m=+26.794345665" watchObservedRunningTime="2026-07-16 15:35:39.093845787 +0000 UTC m=+26.795397678" Jul 16 15:35:39.209938 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.209893 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-gbvcg" podStartSLOduration=3.55685343 podStartE2EDuration="24.209876724s" totalImagesPullingTime="20.653023294s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:39.175579956 +0000 UTC m=+26.877131845" watchObservedRunningTime="2026-07-16 15:35:39.209876724 +0000 UTC m=+26.911428617" Jul 16 15:35:39.227059 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.227017 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-4zz9k" podStartSLOduration=3.677007816 podStartE2EDuration="24.227000628s" totalImagesPullingTime="20.549992812s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:39.226581065 +0000 UTC m=+26.928132956" watchObservedRunningTime="2026-07-16 15:35:39.227000628 +0000 UTC m=+26.928552520" Jul 16 15:35:39.518181 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.518161 2553 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 16 15:35:39.882929 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.882900 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:39.883209 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.883017 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:39.883209 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:39.883028 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:39.883209 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:39.883063 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:39.883209 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:39.883145 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:39.883416 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:39.883255 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:40.067727 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.067680 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-tcz6r" event={"ID":"da228326-59ec-470d-b3e9-d9634725a7b0","Type":"ContainerStarted","Data":"7719c324794249ced67d3736862b0ca4885dd98c2991c9ed9e0acfd441da4cce"} Jul 16 15:35:40.070107 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.070076 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" event={"ID":"fdf2ddabf14f6f15b51026dc5c8b42a8","Type":"ContainerStarted","Data":"0242b94fa17832a6cbe18112dcd00baec3c97ae1afddc6d8a9abb9e59287186e"} Jul 16 15:35:40.073385 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.073359 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" event={"ID":"ca808dec-8a96-4654-878e-734591ee12ed","Type":"ContainerStarted","Data":"6d9faab3e8338ae9ca3d02ffc509aed7df1bb64e54fd619a0332e3372f2b05bd"} Jul 16 15:35:40.105748 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.105696 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-tcz6r" podStartSLOduration=4.5516506450000005 podStartE2EDuration="25.105678253s" totalImagesPullingTime="20.554027608s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:40.088854765 +0000 UTC m=+27.790406656" watchObservedRunningTime="2026-07-16 15:35:40.105678253 +0000 UTC m=+27.807230143" Jul 16 15:35:40.106267 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.106236 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-168.ec2.internal" podStartSLOduration=24.106224607 podStartE2EDuration="24.106224607s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:40.10530971 +0000 UTC m=+27.806861601" watchObservedRunningTime="2026-07-16 15:35:40.106224607 +0000 UTC m=+27.807776498" Jul 16 15:35:40.434770 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.434659 2553 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-16T15:35:39.518176752Z","UUID":"837c6b25-fbdc-49ec-a79b-360884c26bcf","Handler":null,"Name":"","Endpoint":""} Jul 16 15:35:40.436429 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.436405 2553 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 16 15:35:40.436539 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:40.436445 2553 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 16 15:35:41.050259 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.050214 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:41.050762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.050741 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:41.079083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.079059 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:35:41.079560 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.079535 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"34b9659be02c8b913413d88691e7ca5e9f1525b13a9c53a3a38d0e85516da95f"} Jul 16 15:35:41.080136 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.079901 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:41.080242 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.080192 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-gbvcg" Jul 16 15:35:41.882536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.882499 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:41.882732 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.882550 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:41.882732 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:41.882644 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:41.882732 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:41.882662 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:41.882897 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:41.882747 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:41.882897 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:41.882826 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:42.084762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:42.084727 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" event={"ID":"ca808dec-8a96-4654-878e-734591ee12ed","Type":"ContainerStarted","Data":"cc7c53f9e02c8a3dcd8396f3d19abcbaad599de2ab0fd1e254755c07c90d3a20"} Jul 16 15:35:43.882939 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:43.882763 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:43.883528 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:43.882763 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:43.883528 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:43.883014 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:43.883528 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:43.883102 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:43.883528 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:43.882763 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:43.883528 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:43.883213 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:44.092434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.092406 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:35:44.092852 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.092814 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"8514b3500d7d5d18092b6c7e1f557578ddd1976ee2bab135ace12aaf9377d73c"} Jul 16 15:35:44.093197 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.093160 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:44.093313 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.093245 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:44.093313 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.093266 2553 scope.go:117] "RemoveContainer" containerID="2ca5660e48162c26ea84f165896474f57e00455a40091edb2b92bb599fa62e79" Jul 16 15:35:44.093313 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.093293 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:44.094995 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.094976 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="1fb1eafe8dffc04a32ba1ccf848a410b0dfd54f5294a0a781c944e688ed072f1" exitCode=0 Jul 16 15:35:44.095052 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.095006 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"1fb1eafe8dffc04a32ba1ccf848a410b0dfd54f5294a0a781c944e688ed072f1"} Jul 16 15:35:44.110955 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.110931 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:44.111124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.111108 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:35:44.122915 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:44.122876 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-v4hmp" podStartSLOduration=5.900851879 podStartE2EDuration="29.122863517s" totalImagesPullingTime="23.222011638s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:42.107253805 +0000 UTC m=+29.808805730" watchObservedRunningTime="2026-07-16 15:35:44.122863517 +0000 UTC m=+31.824415408" Jul 16 15:35:45.085165 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.085134 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gtpqd"] Jul 16 15:35:45.085790 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.085258 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:45.085790 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:45.085350 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:45.088138 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.088115 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqk7c"] Jul 16 15:35:45.088249 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.088206 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:45.088295 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:45.088279 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:45.090679 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.090658 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-48sll"] Jul 16 15:35:45.090773 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.090745 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:45.090825 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:45.090812 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:45.101606 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.101588 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:35:45.101983 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.101964 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" event={"ID":"6ffde159-037f-4b6e-ab53-58f0e26ecae4","Type":"ContainerStarted","Data":"ea6ecae2fac220f7b60fdc35e6ba753209efe58826017b52d93b733698879585"} Jul 16 15:35:45.130683 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:45.130643 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" podStartSLOduration=9.433489479 podStartE2EDuration="30.1306129s" totalImagesPullingTime="20.697123421s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:45.130259226 +0000 UTC m=+32.831811116" watchObservedRunningTime="2026-07-16 15:35:45.1306129 +0000 UTC m=+32.832164781" Jul 16 15:35:46.883164 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:46.882991 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:46.883487 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:46.883017 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:46.883487 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:46.883244 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:46.883487 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:46.883047 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:46.883487 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:46.883350 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:46.883487 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:46.883434 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:47.107557 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:47.107494 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="9bc6eb07551c07a3ea19024bc1e38bfe0d9d4801c81e26b092cb4cacf3c6b7ce" exitCode=0 Jul 16 15:35:47.107557 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:47.107543 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"9bc6eb07551c07a3ea19024bc1e38bfe0d9d4801c81e26b092cb4cacf3c6b7ce"} Jul 16 15:35:47.790479 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:47.790449 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:47.790669 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.790551 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:47.790669 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.790598 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:19.790584642 +0000 UTC m=+67.492136509 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 16 15:35:47.891703 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:47.891671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:47.892056 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.891807 2553 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:47.892056 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.891859 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret podName:116501b0-acde-4960-b464-465ccd1d27a6 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:19.891846084 +0000 UTC m=+67.593397956 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret") pod "global-pull-secret-syncer-gtpqd" (UID: "116501b0-acde-4960-b464-465ccd1d27a6") : object "kube-system"/"original-pull-secret" not registered Jul 16 15:35:47.992561 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:47.992530 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:47.992715 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.992687 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 16 15:35:47.992715 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.992704 2553 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 16 15:35:47.992715 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.992713 2553 projected.go:196] Error preparing data for projected volume kube-api-access-flcd2 for pod openshift-network-diagnostics/network-check-target-48sll: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:47.992817 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:47.992757 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2 podName:de1997a6-b45d-4e43-aaa7-7bf05e0641e8 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:19.992744872 +0000 UTC m=+67.694296740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-flcd2" (UniqueName: "kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2") pod "network-check-target-48sll" (UID: "de1997a6-b45d-4e43-aaa7-7bf05e0641e8") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 16 15:35:48.885051 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:48.885022 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:48.885198 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:48.885104 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:48.885198 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:48.885113 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gtpqd" podUID="116501b0-acde-4960-b464-465ccd1d27a6" Jul 16 15:35:48.885198 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:48.885127 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:48.885297 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:48.885204 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-48sll" podUID="de1997a6-b45d-4e43-aaa7-7bf05e0641e8" Jul 16 15:35:48.885297 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:48.885262 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqk7c" podUID="87f476dc-4dbe-4b63-ba23-4e6177babe08" Jul 16 15:35:50.115930 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.115908 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="955bf1a96d6eb9eaec3e5dd6880608fd2cbb1776b60d149a7193629c4834bd6d" exitCode=0 Jul 16 15:35:50.116302 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.115950 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"955bf1a96d6eb9eaec3e5dd6880608fd2cbb1776b60d149a7193629c4834bd6d"} Jul 16 15:35:50.617108 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.617084 2553 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-134-168.ec2.internal" event="NodeReady" Jul 16 15:35:50.617252 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.617186 2553 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 16 15:35:50.659940 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.659911 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:35:50.663248 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.663232 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.666258 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.666243 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 16 15:35:50.666463 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.666445 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 16 15:35:50.666544 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.666491 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 16 15:35:50.666739 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.666724 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8gcr9\"" Jul 16 15:35:50.673689 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.673670 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-4qgdz"] Jul 16 15:35:50.674117 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.674098 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 16 15:35:50.677557 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.677537 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:35:50.677681 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.677631 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.680262 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.680213 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 16 15:35:50.680262 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.680229 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vskms\"" Jul 16 15:35:50.680499 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.680289 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 16 15:35:50.686540 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.686525 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-vl47k"] Jul 16 15:35:50.690291 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.690271 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.690521 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.690473 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4qgdz"] Jul 16 15:35:50.692919 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.692900 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 16 15:35:50.693731 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.693712 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 16 15:35:50.693925 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.693908 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-6gbrz\"" Jul 16 15:35:50.694363 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.694342 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 16 15:35:50.703811 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.703792 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vl47k"] Jul 16 15:35:50.812836 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812809 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812836 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812837 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sjzl\" (UniqueName: \"kubernetes.io/projected/29e3315a-0858-414b-9521-47caafd81e3a-kube-api-access-6sjzl\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812874 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812892 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e3315a-0858-414b-9521-47caafd81e3a-tmp-dir\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812919 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812935 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812956 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812973 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.812997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.812994 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-92jkt\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.813214 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.813011 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-cert\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.813214 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.813025 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mxgsr\" (UniqueName: \"kubernetes.io/projected/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-kube-api-access-mxgsr\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.813214 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.813042 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.813214 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.813064 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.813214 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.813079 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/29e3315a-0858-414b-9521-47caafd81e3a-config-volume\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.878147 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.878070 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gbvcg_e3e5d41d-c3d4-423c-88a5-c15a648843dc/konnectivity-agent/0.log" limit=1024 Jul 16 15:35:50.886309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.886290 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:35:50.886309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.886302 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:35:50.886451 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.886297 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:35:50.907536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.907503 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 16 15:35:50.907979 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.907945 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2scpx\"" Jul 16 15:35:50.908206 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.908183 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 15:35:50.908472 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.908452 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 15:35:50.908803 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.908636 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfqhq\"" Jul 16 15:35:50.908803 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.908722 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 16 15:35:50.913636 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913603 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913727 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913647 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6sjzl\" (UniqueName: \"kubernetes.io/projected/29e3315a-0858-414b-9521-47caafd81e3a-kube-api-access-6sjzl\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.913727 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913701 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913727 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913723 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e3315a-0858-414b-9521-47caafd81e3a-tmp-dir\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913753 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913809 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913833 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:50.913841 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:50.913857 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913856 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-92jkt\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.913892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.913880 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-cert\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:50.913933 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:51.41390154 +0000 UTC m=+39.115453427 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914041 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/29e3315a-0858-414b-9521-47caafd81e3a-tmp-dir\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914180 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mxgsr\" (UniqueName: \"kubernetes.io/projected/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-kube-api-access-mxgsr\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914205 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914226 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.914261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914243 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/29e3315a-0858-414b-9521-47caafd81e3a-config-volume\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.914550 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914321 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.914550 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:50.914403 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:35:50.914550 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:50.914461 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:35:51.414445611 +0000 UTC m=+39.115997492 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:35:50.914550 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914543 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.914772 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914747 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/29e3315a-0858-414b-9521-47caafd81e3a-config-volume\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.914926 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.914902 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.918232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.918209 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.918315 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.918233 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-cert\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.918741 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.918723 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.927964 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.927941 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sjzl\" (UniqueName: \"kubernetes.io/projected/29e3315a-0858-414b-9521-47caafd81e3a-kube-api-access-6sjzl\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:50.929724 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.929702 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-92jkt\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.930667 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.930647 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:50.930926 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.930908 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mxgsr\" (UniqueName: \"kubernetes.io/projected/3af65aa6-9f3c-4000-a63e-a03a4134d0cd-kube-api-access-mxgsr\") pod \"ingress-canary-vl47k\" (UID: \"3af65aa6-9f3c-4000-a63e-a03a4134d0cd\") " pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:50.981241 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.981212 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gbvcg_e3e5d41d-c3d4-423c-88a5-c15a648843dc/konnectivity-agent/0.log" limit=1024 Jul 16 15:35:50.999081 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:50.999063 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-vl47k" Jul 16 15:35:51.160066 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:51.159844 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-vl47k"] Jul 16 15:35:51.167283 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:51.167256 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3af65aa6_9f3c_4000_a63e_a03a4134d0cd.slice/crio-b601afcddb9bcff0bfa135e630bf28ae5da96cc555b84f787735977c2be45f46 WatchSource:0}: Error finding container b601afcddb9bcff0bfa135e630bf28ae5da96cc555b84f787735977c2be45f46: Status 404 returned error can't find the container with id b601afcddb9bcff0bfa135e630bf28ae5da96cc555b84f787735977c2be45f46 Jul 16 15:35:51.417944 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:51.417867 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:51.417944 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:51.417934 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:51.418121 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:51.417975 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:35:51.418121 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:51.417992 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:35:51.418121 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:51.418050 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:52.418033839 +0000 UTC m=+40.119585732 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:35:51.418121 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:51.418083 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:35:51.418302 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:51.418139 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:35:52.418124439 +0000 UTC m=+40.119676314 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:35:52.122860 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:52.122738 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vl47k" event={"ID":"3af65aa6-9f3c-4000-a63e-a03a4134d0cd","Type":"ContainerStarted","Data":"b601afcddb9bcff0bfa135e630bf28ae5da96cc555b84f787735977c2be45f46"} Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:52.425252 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:52.425319 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:52.425470 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:52.425533 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:35:54.425513483 +0000 UTC m=+42.127065365 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:52.425677 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:52.425693 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:35:52.425821 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:52.425744 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:54.42572981 +0000 UTC m=+42.127281680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:35:54.443317 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:54.443285 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:54.443385 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:54.443452 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:54.443486 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:54.443499 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:54.443517 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:35:58.443495387 +0000 UTC m=+46.145047257 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:35:54.444020 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:54.443541 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:58.443526265 +0000 UTC m=+46.145078133 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:35:55.875347 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:55.875317 2553 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-gbvcg_e3e5d41d-c3d4-423c-88a5-c15a648843dc/konnectivity-agent/0.log" limit=1024 Jul 16 15:35:56.135314 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:56.135239 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="6d82f55f5ac11f5e48c530c8b09524d406803204d9d7c4078c3d485d531d0b6b" exitCode=0 Jul 16 15:35:56.135452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:56.135311 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"6d82f55f5ac11f5e48c530c8b09524d406803204d9d7c4078c3d485d531d0b6b"} Jul 16 15:35:56.136677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:56.136654 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-vl47k" event={"ID":"3af65aa6-9f3c-4000-a63e-a03a4134d0cd","Type":"ContainerStarted","Data":"91f7e2027f500d66eee775683bd95fcc067fcd23de714a73570681b65feb987c"} Jul 16 15:35:56.180028 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:56.179968 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-vl47k" podStartSLOduration=1.87971835 podStartE2EDuration="6.17994872s" totalImagesPullingTime="4.30023037s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:56.179923723 +0000 UTC m=+43.881475614" watchObservedRunningTime="2026-07-16 15:35:56.17994872 +0000 UTC m=+43.881500612" Jul 16 15:35:57.141789 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:57.141756 2553 generic.go:356] "Generic (PLEG): container finished" podID="aacaf418-8d1b-4748-af53-3c57864d3ddf" containerID="f98e6b6e5014ad52a488991f41d4fc26a2e605bc38b8246f4ceeb35a966e894a" exitCode=0 Jul 16 15:35:57.142179 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:57.141843 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerDied","Data":"f98e6b6e5014ad52a488991f41d4fc26a2e605bc38b8246f4ceeb35a966e894a"} Jul 16 15:35:58.147098 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.147067 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-r4749" event={"ID":"aacaf418-8d1b-4748-af53-3c57864d3ddf","Type":"ContainerStarted","Data":"8372ea76ad64b6fa61de68b005ba47afb014ba8dc4118c2ebf27e6ac2d22af47"} Jul 16 15:35:58.173972 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.173925 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-r4749" podStartSLOduration=8.300706145 podStartE2EDuration="43.173910986s" totalImagesPullingTime="34.873204841s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:35:58.173303805 +0000 UTC m=+45.874855695" watchObservedRunningTime="2026-07-16 15:35:58.173910986 +0000 UTC m=+45.875462876" Jul 16 15:35:58.394261 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.394230 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6"] Jul 16 15:35:58.417664 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.417585 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-75bd6b5b78-gc5mg"] Jul 16 15:35:58.417782 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.417708 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.420378 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.420356 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rnngt\"" Jul 16 15:35:58.420501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.420356 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 16 15:35:58.420501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.420436 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.421733 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.421713 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.421828 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.421749 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 16 15:35:58.437687 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.437667 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fzp89"] Jul 16 15:35:58.437777 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.437768 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.440147 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440128 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 16 15:35:58.440346 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440333 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.440413 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440386 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 16 15:35:58.440464 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440443 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 16 15:35:58.440521 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440440 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.440575 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440526 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 16 15:35:58.440644 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.440581 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rr5xg\"" Jul 16 15:35:58.457806 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.457788 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6"] Jul 16 15:35:58.457806 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.457800 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.457908 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.457830 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fzp89"] Jul 16 15:35:58.457908 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.457846 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75bd6b5b78-gc5mg"] Jul 16 15:35:58.460577 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.460545 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\"" Jul 16 15:35:58.460577 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.460576 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.461023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.461008 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.461091 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.461065 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\"" Jul 16 15:35:58.461091 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.461081 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-4vs6k\"" Jul 16 15:35:58.466821 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.466801 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\"" Jul 16 15:35:58.473138 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473120 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kq7gx\" (UniqueName: \"kubernetes.io/projected/a99c00fc-e6ba-450a-9505-c614036a222c-kube-api-access-kq7gx\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.473240 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473150 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-default-certificate\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.473240 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473200 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dv62x\" (UniqueName: \"kubernetes.io/projected/59387a74-b93a-49b7-86a6-4a6223fb9191-kube-api-access-dv62x\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.473240 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473226 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-snapshots\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.473377 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473269 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:35:58.473377 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473296 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.473377 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473339 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.473374 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.473389 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.473427 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.473433 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.473419215 +0000 UTC m=+54.174971083 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473374 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.473459 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.473449312 +0000 UTC m=+54.175001180 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:35:58.473481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473481 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473502 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018f2da5-c589-4c28-988f-432c9643da7c-serving-cert\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473518 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a99c00fc-e6ba-450a-9505-c614036a222c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473569 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg9w5\" (UniqueName: \"kubernetes.io/projected/018f2da5-c589-4c28-988f-432c9643da7c-kube-api-access-dg9w5\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473602 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473653 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-stats-auth\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473678 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.473788 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.473742 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-tmp\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.490215 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.490192 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl"] Jul 16 15:35:58.509833 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.509805 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp"] Jul 16 15:35:58.514683 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.513248 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" Jul 16 15:35:58.516006 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.515984 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-q6s9f\"" Jul 16 15:35:58.523857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.523838 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2"] Jul 16 15:35:58.523955 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.523942 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" Jul 16 15:35:58.526518 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.526500 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.526653 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.526635 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.527079 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.527058 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-r6n7d\"" Jul 16 15:35:58.557925 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.557897 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl"] Jul 16 15:35:58.557925 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.557930 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp"] Jul 16 15:35:58.558101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.557947 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs"] Jul 16 15:35:58.558101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.558028 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.560710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.560683 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\"" Jul 16 15:35:58.560972 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.560949 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-hqhsl\"" Jul 16 15:35:58.561083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.560973 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\"" Jul 16 15:35:58.561083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.560981 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.561083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.560974 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.574662 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574643 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kq7gx\" (UniqueName: \"kubernetes.io/projected/a99c00fc-e6ba-450a-9505-c614036a222c-kube-api-access-kq7gx\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.574746 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574671 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-default-certificate\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.574746 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574692 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hzlz2\" (UniqueName: \"kubernetes.io/projected/bba67bb0-2159-4951-938e-cd1c782b6af4-kube-api-access-hzlz2\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.574746 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574710 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bba67bb0-2159-4951-938e-cd1c782b6af4-config\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.574842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574769 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dv62x\" (UniqueName: \"kubernetes.io/projected/59387a74-b93a-49b7-86a6-4a6223fb9191-kube-api-access-dv62x\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.574842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-snapshots\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.574842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574834 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sd59g\" (UniqueName: \"kubernetes.io/projected/80c6cade-892f-4f61-9712-53a749f66d26-kube-api-access-sd59g\") pod \"volume-data-source-validator-78495597f6-mh6lp\" (UID: \"80c6cade-892f-4f61-9712-53a749f66d26\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" Jul 16 15:35:58.574939 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574888 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bba67bb0-2159-4951-938e-cd1c782b6af4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.574939 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574917 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.575040 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.574954 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575040 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.574995 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:35:58.575040 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575009 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.575185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575042 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b9xrn\" (UniqueName: \"kubernetes.io/projected/7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6-kube-api-access-b9xrn\") pod \"network-check-source-869dcfcdcd-vqjfl\" (UID: \"7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" Jul 16 15:35:58.575185 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.575056 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:35:59.075036431 +0000 UTC m=+46.776588299 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:35:58.575185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575091 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018f2da5-c589-4c28-988f-432c9643da7c-serving-cert\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575185 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.575119 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:59.075102986 +0000 UTC m=+46.776654855 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:35:58.575185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575153 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a99c00fc-e6ba-450a-9505-c614036a222c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.575433 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575192 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dg9w5\" (UniqueName: \"kubernetes.io/projected/018f2da5-c589-4c28-988f-432c9643da7c-kube-api-access-dg9w5\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575433 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575225 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.575433 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-stats-auth\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.575433 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575288 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575433 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575327 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-tmp\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575702 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575467 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-snapshots\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575702 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575582 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/018f2da5-c589-4c28-988f-432c9643da7c-tmp\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575702 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.575683 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:35:58.575832 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.575725 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:35:59.07571138 +0000 UTC m=+46.777263253 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:35:58.575891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575865 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-trusted-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575932 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575911 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/018f2da5-c589-4c28-988f-432c9643da7c-service-ca-bundle\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.575985 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.575961 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/a99c00fc-e6ba-450a-9505-c614036a222c-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.577636 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.577601 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-default-certificate\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.577757 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.577737 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/018f2da5-c589-4c28-988f-432c9643da7c-serving-cert\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.577978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.577942 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-stats-auth\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.582042 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.582022 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2"] Jul 16 15:35:58.582152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.582055 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs"] Jul 16 15:35:58.582152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.582119 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.585299 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.585281 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\"" Jul 16 15:35:58.585642 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.585610 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.585740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.585643 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\"" Jul 16 15:35:58.585740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.585672 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.585740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.585695 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-gdcf9\"" Jul 16 15:35:58.591124 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.591107 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg9w5\" (UniqueName: \"kubernetes.io/projected/018f2da5-c589-4c28-988f-432c9643da7c-kube-api-access-dg9w5\") pod \"insights-operator-68d9dd94cf-fzp89\" (UID: \"018f2da5-c589-4c28-988f-432c9643da7c\") " pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.594843 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.594819 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dv62x\" (UniqueName: \"kubernetes.io/projected/59387a74-b93a-49b7-86a6-4a6223fb9191-kube-api-access-dv62x\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:58.595225 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.595206 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kq7gx\" (UniqueName: \"kubernetes.io/projected/a99c00fc-e6ba-450a-9505-c614036a222c-kube-api-access-kq7gx\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:58.599309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.599293 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh"] Jul 16 15:35:58.630473 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.630450 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh"] Jul 16 15:35:58.630557 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.630481 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7866497494-d4qkh"] Jul 16 15:35:58.630589 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.630557 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.633185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.633159 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\"" Jul 16 15:35:58.633185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.633168 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.633348 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.633190 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-795xs\"" Jul 16 15:35:58.633348 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.633233 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.644355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.644334 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-d4qkh"] Jul 16 15:35:58.644441 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.644405 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.647018 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.647000 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:35:58.647123 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.647044 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 16 15:35:58.647123 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.647063 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-sd2fz\"" Jul 16 15:35:58.647123 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.647067 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 16 15:35:58.647380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.647364 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 16 15:35:58.660324 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.660300 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676482 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676605 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-config\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676674 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2fe278-4fec-4892-80d4-70adfb9e3e44-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676719 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hzlz2\" (UniqueName: \"kubernetes.io/projected/bba67bb0-2159-4951-938e-cd1c782b6af4-kube-api-access-hzlz2\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676751 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d9wnf\" (UniqueName: \"kubernetes.io/projected/2f2fe278-4fec-4892-80d4-70adfb9e3e44-kube-api-access-d9wnf\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676786 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bba67bb0-2159-4951-938e-cd1c782b6af4-config\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676815 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5a56410-8aa7-414a-b2ac-3614a129bf35-serving-cert\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676857 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4972n\" (UniqueName: \"kubernetes.io/projected/c5a56410-8aa7-414a-b2ac-3614a129bf35-kube-api-access-4972n\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676897 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-sd59g\" (UniqueName: \"kubernetes.io/projected/80c6cade-892f-4f61-9712-53a749f66d26-kube-api-access-sd59g\") pod \"volume-data-source-validator-78495597f6-mh6lp\" (UID: \"80c6cade-892f-4f61-9712-53a749f66d26\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.676982 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bba67bb0-2159-4951-938e-cd1c782b6af4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.677029 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-trusted-ca\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.677152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.677065 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4j5rk\" (UniqueName: \"kubernetes.io/projected/18182678-feea-4569-94de-ef4be91e5a9e-kube-api-access-4j5rk\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.680014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.677151 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b9xrn\" (UniqueName: \"kubernetes.io/projected/7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6-kube-api-access-b9xrn\") pod \"network-check-source-869dcfcdcd-vqjfl\" (UID: \"7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" Jul 16 15:35:58.680014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.677193 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2fe278-4fec-4892-80d4-70adfb9e3e44-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.680014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.678148 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/bba67bb0-2159-4951-938e-cd1c782b6af4-config\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.680819 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.680801 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/bba67bb0-2159-4951-938e-cd1c782b6af4-serving-cert\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.686537 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.686511 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-sd59g\" (UniqueName: \"kubernetes.io/projected/80c6cade-892f-4f61-9712-53a749f66d26-kube-api-access-sd59g\") pod \"volume-data-source-validator-78495597f6-mh6lp\" (UID: \"80c6cade-892f-4f61-9712-53a749f66d26\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" Jul 16 15:35:58.686615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.686594 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hzlz2\" (UniqueName: \"kubernetes.io/projected/bba67bb0-2159-4951-938e-cd1c782b6af4-kube-api-access-hzlz2\") pod \"service-ca-operator-7695fc6c8c-d7vd2\" (UID: \"bba67bb0-2159-4951-938e-cd1c782b6af4\") " pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.688010 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.687994 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b9xrn\" (UniqueName: \"kubernetes.io/projected/7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6-kube-api-access-b9xrn\") pod \"network-check-source-869dcfcdcd-vqjfl\" (UID: \"7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6\") " pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" Jul 16 15:35:58.766593 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.766575 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" Jul 16 15:35:58.777998 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.777975 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-config\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.778070 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778013 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2fe278-4fec-4892-80d4-70adfb9e3e44-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.778070 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778041 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d9wnf\" (UniqueName: \"kubernetes.io/projected/2f2fe278-4fec-4892-80d4-70adfb9e3e44-kube-api-access-d9wnf\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.778180 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778072 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5a56410-8aa7-414a-b2ac-3614a129bf35-serving-cert\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.778180 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778109 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4972n\" (UniqueName: \"kubernetes.io/projected/c5a56410-8aa7-414a-b2ac-3614a129bf35-kube-api-access-4972n\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.778315 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778292 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-trusted-ca\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.778369 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778339 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4j5rk\" (UniqueName: \"kubernetes.io/projected/18182678-feea-4569-94de-ef4be91e5a9e-kube-api-access-4j5rk\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.778423 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778407 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2fe278-4fec-4892-80d4-70adfb9e3e44-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.778472 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778459 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.778535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778512 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/2f2fe278-4fec-4892-80d4-70adfb9e3e44-config\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.778691 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.778616 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:35:58.778827 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.778696 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-config\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.778827 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:58.778708 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:35:59.278690595 +0000 UTC m=+46.980242466 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:35:58.779263 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.779239 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/c5a56410-8aa7-414a-b2ac-3614a129bf35-trusted-ca\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.780350 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.780331 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/2f2fe278-4fec-4892-80d4-70adfb9e3e44-serving-cert\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.780418 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.780352 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/c5a56410-8aa7-414a-b2ac-3614a129bf35-serving-cert\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.787397 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.787372 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d9wnf\" (UniqueName: \"kubernetes.io/projected/2f2fe278-4fec-4892-80d4-70adfb9e3e44-kube-api-access-d9wnf\") pod \"kube-storage-version-migrator-operator-799cb7f958-npbbs\" (UID: \"2f2fe278-4fec-4892-80d4-70adfb9e3e44\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.787481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.787417 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4972n\" (UniqueName: \"kubernetes.io/projected/c5a56410-8aa7-414a-b2ac-3614a129bf35-kube-api-access-4972n\") pod \"console-operator-7866497494-d4qkh\" (UID: \"c5a56410-8aa7-414a-b2ac-3614a129bf35\") " pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.787746 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.787729 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4j5rk\" (UniqueName: \"kubernetes.io/projected/18182678-feea-4569-94de-ef4be91e5a9e-kube-api-access-4j5rk\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:58.822795 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.822770 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" Jul 16 15:35:58.850385 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.850363 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" Jul 16 15:35:58.866965 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.866943 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" Jul 16 15:35:58.904560 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.904535 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" Jul 16 15:35:58.909752 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.909719 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-68d9dd94cf-fzp89"] Jul 16 15:35:58.916187 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:58.916157 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod018f2da5_c589_4c28_988f_432c9643da7c.slice/crio-88430f18c1362c67aecdc72419d567b4e5c17a03e8d2960d48560f2fd20bbd9a WatchSource:0}: Error finding container 88430f18c1362c67aecdc72419d567b4e5c17a03e8d2960d48560f2fd20bbd9a: Status 404 returned error can't find the container with id 88430f18c1362c67aecdc72419d567b4e5c17a03e8d2960d48560f2fd20bbd9a Jul 16 15:35:58.952550 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.952414 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:35:58.991071 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:58.991037 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl"] Jul 16 15:35:59.038243 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.038187 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp"] Jul 16 15:35:59.043645 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:59.043600 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80c6cade_892f_4f61_9712_53a749f66d26.slice/crio-773df0e43e542ac52ef501b16c4d443a11047d8e0a75af899183ce1a683a483f WatchSource:0}: Error finding container 773df0e43e542ac52ef501b16c4d443a11047d8e0a75af899183ce1a683a483f: Status 404 returned error can't find the container with id 773df0e43e542ac52ef501b16c4d443a11047d8e0a75af899183ce1a683a483f Jul 16 15:35:59.070707 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.070678 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs"] Jul 16 15:35:59.080775 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.080755 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:35:59.080867 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.080794 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:59.080913 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.080902 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:35:59.080944 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.080921 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:35:59.081003 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.080992 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:00.080972949 +0000 UTC m=+47.782524834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:35:59.081056 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.081031 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:35:59.081089 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.081070 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:00.081053609 +0000 UTC m=+47.782605494 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:35:59.081089 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.081088 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:00.081080972 +0000 UTC m=+47.782632839 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:35:59.083853 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.083791 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2"] Jul 16 15:35:59.090042 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:59.090021 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbba67bb0_2159_4951_938e_cd1c782b6af4.slice/crio-2cb79c8535006373cfaf871b2ab739bba0b0b5dca1643c232a28523328834dbd WatchSource:0}: Error finding container 2cb79c8535006373cfaf871b2ab739bba0b0b5dca1643c232a28523328834dbd: Status 404 returned error can't find the container with id 2cb79c8535006373cfaf871b2ab739bba0b0b5dca1643c232a28523328834dbd Jul 16 15:35:59.090712 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:59.090688 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f2fe278_4fec_4892_80d4_70adfb9e3e44.slice/crio-2524914b4ec7c02a87b5a536c83c76e76aa4de40b6e0951d4a7d53a31d533588 WatchSource:0}: Error finding container 2524914b4ec7c02a87b5a536c83c76e76aa4de40b6e0951d4a7d53a31d533588: Status 404 returned error can't find the container with id 2524914b4ec7c02a87b5a536c83c76e76aa4de40b6e0951d4a7d53a31d533588 Jul 16 15:35:59.131451 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.131428 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7866497494-d4qkh"] Jul 16 15:35:59.134823 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:35:59.134802 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc5a56410_8aa7_414a_b2ac_3614a129bf35.slice/crio-d0d888c4a04ea4753566a26fc8e6bdf37534b36d01ac92462d30f5d16d3e3695 WatchSource:0}: Error finding container d0d888c4a04ea4753566a26fc8e6bdf37534b36d01ac92462d30f5d16d3e3695: Status 404 returned error can't find the container with id d0d888c4a04ea4753566a26fc8e6bdf37534b36d01ac92462d30f5d16d3e3695 Jul 16 15:35:59.150615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.150585 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" event={"ID":"80c6cade-892f-4f61-9712-53a749f66d26","Type":"ContainerStarted","Data":"773df0e43e542ac52ef501b16c4d443a11047d8e0a75af899183ce1a683a483f"} Jul 16 15:35:59.151676 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.151652 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" event={"ID":"7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6","Type":"ContainerStarted","Data":"0f3bb7bd5dda6aff1a0f68b9145873f4a1f6bde476403947aab8abffac3c129f"} Jul 16 15:35:59.152723 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.152704 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" event={"ID":"018f2da5-c589-4c28-988f-432c9643da7c","Type":"ContainerStarted","Data":"88430f18c1362c67aecdc72419d567b4e5c17a03e8d2960d48560f2fd20bbd9a"} Jul 16 15:35:59.153655 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.153637 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d4qkh" event={"ID":"c5a56410-8aa7-414a-b2ac-3614a129bf35","Type":"ContainerStarted","Data":"d0d888c4a04ea4753566a26fc8e6bdf37534b36d01ac92462d30f5d16d3e3695"} Jul 16 15:35:59.154594 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.154568 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" event={"ID":"2f2fe278-4fec-4892-80d4-70adfb9e3e44","Type":"ContainerStarted","Data":"2524914b4ec7c02a87b5a536c83c76e76aa4de40b6e0951d4a7d53a31d533588"} Jul 16 15:35:59.155485 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.155463 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" event={"ID":"bba67bb0-2159-4951-938e-cd1c782b6af4","Type":"ContainerStarted","Data":"2cb79c8535006373cfaf871b2ab739bba0b0b5dca1643c232a28523328834dbd"} Jul 16 15:35:59.282024 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:35:59.281956 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:35:59.282115 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.282088 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:35:59.282161 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:35:59.282139 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:36:00.282125086 +0000 UTC m=+47.983676954 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:36:00.088741 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:00.088705 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:00.088936 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:00.088849 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:00.088936 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.088876 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:36:00.088936 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:00.088905 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:00.089117 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.088944 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:02.088929456 +0000 UTC m=+49.790481323 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:36:00.089117 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.089039 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:02.08902372 +0000 UTC m=+49.790575590 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:36:00.089117 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.089116 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:00.089274 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.089149 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:02.089138602 +0000 UTC m=+49.790690472 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:00.290127 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:00.290088 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:00.290583 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.290248 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:36:00.290583 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:00.290319 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:36:02.290297496 +0000 UTC m=+49.991849368 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:36:02.108697 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:02.108635 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.108806 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:02.108800 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.108892 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.108866846 +0000 UTC m=+53.810418728 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.108910 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:02.108962 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.108970 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.108954547 +0000 UTC m=+53.810506424 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:02.109269 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.109069 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.109027922 +0000 UTC m=+53.810579800 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:36:02.310889 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:02.310854 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:02.311058 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.311014 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:36:02.311126 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:02.311096 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:36:06.311074564 +0000 UTC m=+54.012626459 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:36:06.145321 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.145229 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.145379 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.145434 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.145439 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.145539 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.145507154 +0000 UTC m=+61.847059037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.145559 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.145580 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.145565952 +0000 UTC m=+61.847117826 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:36:06.145775 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.145611 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.14559688 +0000 UTC m=+61.847148763 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:06.186577 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.186543 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" event={"ID":"018f2da5-c589-4c28-988f-432c9643da7c","Type":"ContainerStarted","Data":"cd09a118ac4a8b93a79e19a5fcfd48198a033c1f7a9bf0ab4176a702035d41eb"} Jul 16 15:36:06.188330 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.188284 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/0.log" Jul 16 15:36:06.188481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.188332 2553 generic.go:356] "Generic (PLEG): container finished" podID="c5a56410-8aa7-414a-b2ac-3614a129bf35" containerID="cb7acf5b444791fab8d2c0ff4c4bce435d4619f16db2bd4f4149b38b68784702" exitCode=255 Jul 16 15:36:06.188481 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.188401 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d4qkh" event={"ID":"c5a56410-8aa7-414a-b2ac-3614a129bf35","Type":"ContainerDied","Data":"cb7acf5b444791fab8d2c0ff4c4bce435d4619f16db2bd4f4149b38b68784702"} Jul 16 15:36:06.188646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.188599 2553 scope.go:117] "RemoveContainer" containerID="cb7acf5b444791fab8d2c0ff4c4bce435d4619f16db2bd4f4149b38b68784702" Jul 16 15:36:06.190248 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.190219 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" event={"ID":"2f2fe278-4fec-4892-80d4-70adfb9e3e44","Type":"ContainerStarted","Data":"082e97de54de5a20bcd6cb9db61c384c201bbc3c1d3a583ebee4330379075ced"} Jul 16 15:36:06.192100 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.192081 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" event={"ID":"bba67bb0-2159-4951-938e-cd1c782b6af4","Type":"ContainerStarted","Data":"5d82514522591940cd04e91939a363580fdb2333545882af6e25e3918d6f3d84"} Jul 16 15:36:06.193541 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.193513 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" event={"ID":"80c6cade-892f-4f61-9712-53a749f66d26","Type":"ContainerStarted","Data":"0d36788d896bfe1aa78b78948b74dd78a066452f3301dfec788ba5c3bf08e40f"} Jul 16 15:36:06.195151 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.195118 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" event={"ID":"7c77caf9-e7c3-407f-9de8-8cc28c1ed9f6","Type":"ContainerStarted","Data":"7c7a996ded9c42c2c5ea07d051b34422219aeb80dc844d66277ee9871911a452"} Jul 16 15:36:06.205684 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.205639 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" podStartSLOduration=1.5603750299999999 podStartE2EDuration="8.205607191s" totalImagesPullingTime="6.645232161s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:06.204939668 +0000 UTC m=+53.906491559" watchObservedRunningTime="2026-07-16 15:36:06.205607191 +0000 UTC m=+53.907159082" Jul 16 15:36:06.239100 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.239054 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-869dcfcdcd-vqjfl" podStartSLOduration=1.580294339 podStartE2EDuration="8.239035104s" totalImagesPullingTime="6.658740765s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:06.237435351 +0000 UTC m=+53.938987240" watchObservedRunningTime="2026-07-16 15:36:06.239035104 +0000 UTC m=+53.940586995" Jul 16 15:36:06.272370 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.272307 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" podStartSLOduration=1.716591099 podStartE2EDuration="8.272283153s" totalImagesPullingTime="6.555692054s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:06.271919486 +0000 UTC m=+53.973471377" watchObservedRunningTime="2026-07-16 15:36:06.272283153 +0000 UTC m=+53.973835044" Jul 16 15:36:06.273914 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.273877 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" podStartSLOduration=1.71807826 podStartE2EDuration="8.273865287s" totalImagesPullingTime="6.555787027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:06.255095575 +0000 UTC m=+53.956647465" watchObservedRunningTime="2026-07-16 15:36:06.273865287 +0000 UTC m=+53.975417173" Jul 16 15:36:06.294884 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.294727 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-mh6lp" podStartSLOduration=1.6924044710000001 podStartE2EDuration="8.294710454s" totalImagesPullingTime="6.602305983s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:06.294077366 +0000 UTC m=+53.995629254" watchObservedRunningTime="2026-07-16 15:36:06.294710454 +0000 UTC m=+53.996262346" Jul 16 15:36:06.347971 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.347854 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:06.348118 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.348108 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:36:06.348174 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.348163 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.3481461 +0000 UTC m=+62.049697989 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:36:06.550195 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.550114 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:06.550195 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.550182 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:06.550400 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.550285 2553 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 16 15:36:06.550400 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.550309 2553 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9fcff7dcb-gsz5j: secret "image-registry-tls" not found Jul 16 15:36:06.550400 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.550342 2553 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 16 15:36:06.550400 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.550368 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls podName:0260ce11-3482-4709-9e13-ba6c8635c106 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:22.550348987 +0000 UTC m=+70.251900858 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls") pod "image-registry-9fcff7dcb-gsz5j" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106") : secret "image-registry-tls" not found Jul 16 15:36:06.550400 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.550399 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls podName:29e3315a-0858-414b-9521-47caafd81e3a nodeName:}" failed. No retries permitted until 2026-07-16 15:36:22.550382896 +0000 UTC m=+70.251934765 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls") pod "dns-default-4qgdz" (UID: "29e3315a-0858-414b-9521-47caafd81e3a") : secret "dns-default-metrics-tls" not found Jul 16 15:36:06.767979 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.767943 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-fpmjg"] Jul 16 15:36:06.790067 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.790034 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fpmjg"] Jul 16 15:36:06.790241 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.790168 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.796241 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.796215 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 16 15:36:06.797564 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.797542 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzsd4\"" Jul 16 15:36:06.797688 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.797663 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 16 15:36:06.852589 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.852507 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b974907-cb45-4089-be70-974082496bba-data-volume\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.852589 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.852555 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.852793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.852685 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.852793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.852754 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b974907-cb45-4089-be70-974082496bba-crio-socket\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.852887 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.852830 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f66kx\" (UniqueName: \"kubernetes.io/projected/2b974907-cb45-4089-be70-974082496bba-kube-api-access-f66kx\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.868695 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.868668 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z"] Jul 16 15:36:06.887902 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.887878 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z"] Jul 16 15:36:06.888014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.887947 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" Jul 16 15:36:06.890723 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.890702 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-t575j\"" Jul 16 15:36:06.890884 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.890705 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 16 15:36:06.891227 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.890747 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 16 15:36:06.953803 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.953776 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.953925 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.953855 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b974907-cb45-4089-be70-974082496bba-crio-socket\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.953925 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.953912 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:06.954040 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.953936 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f66kx\" (UniqueName: \"kubernetes.io/projected/2b974907-cb45-4089-be70-974082496bba-kube-api-access-f66kx\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.954040 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.953970 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:07.453953644 +0000 UTC m=+55.155505512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:06.954040 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.954022 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b974907-cb45-4089-be70-974082496bba-data-volume\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.954197 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.954052 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xzs97\" (UniqueName: \"kubernetes.io/projected/629b55ca-365f-4037-86a4-f0698742a91c-kube-api-access-xzs97\") pod \"migrator-7f59b54f6c-n265z\" (UID: \"629b55ca-365f-4037-86a4-f0698742a91c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" Jul 16 15:36:06.954197 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.954064 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/2b974907-cb45-4089-be70-974082496bba-crio-socket\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.954197 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.954080 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.954197 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.954157 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 15:36:06.954197 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:06.954192 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:07.454183319 +0000 UTC m=+55.155735187 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : configmap "kube-rbac-proxy" not found Jul 16 15:36:06.954432 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.954315 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/2b974907-cb45-4089-be70-974082496bba-data-volume\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:06.962464 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:06.962444 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f66kx\" (UniqueName: \"kubernetes.io/projected/2b974907-cb45-4089-be70-974082496bba-kube-api-access-f66kx\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:07.055293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.055262 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xzs97\" (UniqueName: \"kubernetes.io/projected/629b55ca-365f-4037-86a4-f0698742a91c-kube-api-access-xzs97\") pod \"migrator-7f59b54f6c-n265z\" (UID: \"629b55ca-365f-4037-86a4-f0698742a91c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" Jul 16 15:36:07.064044 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.064021 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xzs97\" (UniqueName: \"kubernetes.io/projected/629b55ca-365f-4037-86a4-f0698742a91c-kube-api-access-xzs97\") pod \"migrator-7f59b54f6c-n265z\" (UID: \"629b55ca-365f-4037-86a4-f0698742a91c\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" Jul 16 15:36:07.198676 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.198649 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" Jul 16 15:36:07.204882 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.204859 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:36:07.205349 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.205330 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/0.log" Jul 16 15:36:07.205460 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.205371 2553 generic.go:356] "Generic (PLEG): container finished" podID="c5a56410-8aa7-414a-b2ac-3614a129bf35" containerID="05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa" exitCode=255 Jul 16 15:36:07.205519 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.205463 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d4qkh" event={"ID":"c5a56410-8aa7-414a-b2ac-3614a129bf35","Type":"ContainerDied","Data":"05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa"} Jul 16 15:36:07.205574 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.205545 2553 scope.go:117] "RemoveContainer" containerID="cb7acf5b444791fab8d2c0ff4c4bce435d4619f16db2bd4f4149b38b68784702" Jul 16 15:36:07.205829 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.205809 2553 scope.go:117] "RemoveContainer" containerID="05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa" Jul 16 15:36:07.205992 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:07.205973 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d4qkh_openshift-console-operator(c5a56410-8aa7-414a-b2ac-3614a129bf35)\"" pod="openshift-console-operator/console-operator-7866497494-d4qkh" podUID="c5a56410-8aa7-414a-b2ac-3614a129bf35" Jul 16 15:36:07.350681 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.350655 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z"] Jul 16 15:36:07.353062 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:07.353035 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod629b55ca_365f_4037_86a4_f0698742a91c.slice/crio-c6f8bbfc7bcdef9c09b27d42978fe49d1e520036e6047cd4a83d5861c759f9ca WatchSource:0}: Error finding container c6f8bbfc7bcdef9c09b27d42978fe49d1e520036e6047cd4a83d5861c759f9ca: Status 404 returned error can't find the container with id c6f8bbfc7bcdef9c09b27d42978fe49d1e520036e6047cd4a83d5861c759f9ca Jul 16 15:36:07.460804 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.460716 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:07.460952 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:07.460804 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:07.460952 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:07.460832 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 15:36:07.460952 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:07.460891 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:08.460877309 +0000 UTC m=+56.162429176 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : configmap "kube-rbac-proxy" not found Jul 16 15:36:07.460952 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:07.460944 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:07.461100 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:07.461005 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:08.460989981 +0000 UTC m=+56.162541869 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:08.171640 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.171605 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z96xk_84ea9286-2f1d-4d42-848e-e48204a86073/dns-node-resolver/0.log" Jul 16 15:36:08.209981 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.209950 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" event={"ID":"629b55ca-365f-4037-86a4-f0698742a91c","Type":"ContainerStarted","Data":"c6f8bbfc7bcdef9c09b27d42978fe49d1e520036e6047cd4a83d5861c759f9ca"} Jul 16 15:36:08.211364 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.211346 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:36:08.211672 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.211657 2553 scope.go:117] "RemoveContainer" containerID="05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa" Jul 16 15:36:08.211813 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:08.211798 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d4qkh_openshift-console-operator(c5a56410-8aa7-414a-b2ac-3614a129bf35)\"" pod="openshift-console-operator/console-operator-7866497494-d4qkh" podUID="c5a56410-8aa7-414a-b2ac-3614a129bf35" Jul 16 15:36:08.470290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.470219 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:08.470290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.470283 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:08.470450 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:08.470340 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 15:36:08.470450 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:08.470406 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.470386831 +0000 UTC m=+58.171938719 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : configmap "kube-rbac-proxy" not found Jul 16 15:36:08.470450 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:08.470414 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:08.470450 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:08.470446 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:10.470437129 +0000 UTC m=+58.171988996 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:08.769460 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.769388 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4zz9k_8338837b-8db4-4083-bab5-0a846db22992/node-ca/0.log" Jul 16 15:36:08.953360 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.953333 2553 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:36:08.953499 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:08.953380 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:36:09.214640 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:09.214597 2553 scope.go:117] "RemoveContainer" containerID="05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa" Jul 16 15:36:09.215045 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:09.214772 2553 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7866497494-d4qkh_openshift-console-operator(c5a56410-8aa7-414a-b2ac-3614a129bf35)\"" pod="openshift-console-operator/console-operator-7866497494-d4qkh" podUID="c5a56410-8aa7-414a-b2ac-3614a129bf35" Jul 16 15:36:09.576306 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:09.576228 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vl47k_3af65aa6-9f3c-4000-a63e-a03a4134d0cd/serve-healthcheck-canary/0.log" Jul 16 15:36:10.219219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:10.219185 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" event={"ID":"629b55ca-365f-4037-86a4-f0698742a91c","Type":"ContainerStarted","Data":"4f4f735a2881310c44a970c54eb9e369456a2f58f22be13185effa86064f6af6"} Jul 16 15:36:10.219581 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:10.219236 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" event={"ID":"629b55ca-365f-4037-86a4-f0698742a91c","Type":"ContainerStarted","Data":"110fe1b91b958746f98d489342bb67580b72bb7b119018912c2a7c71414f89b2"} Jul 16 15:36:10.240728 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:10.240681 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-n265z" podStartSLOduration=1.846555019 podStartE2EDuration="4.240665111s" totalImagesPullingTime="2.394110092s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:10.239748546 +0000 UTC m=+57.941300449" watchObservedRunningTime="2026-07-16 15:36:10.240665111 +0000 UTC m=+57.942217000" Jul 16 15:36:10.488971 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:10.488886 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:10.489133 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:10.489038 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:10.489133 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:10.489097 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.48908209 +0000 UTC m=+62.190633958 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:10.489133 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:10.489098 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 15:36:10.489281 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:10.489148 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:14.489134236 +0000 UTC m=+62.190686104 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : configmap "kube-rbac-proxy" not found Jul 16 15:36:10.489281 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:10.489037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:14.223100 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.223066 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.223125 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.223246 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:30.223230152 +0000 UTC m=+77.924782026 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : configmap references non-existent config key: service-ca.crt Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.223288 2553 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.223351 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.223361 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs podName:59387a74-b93a-49b7-86a6-4a6223fb9191 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:30.223347458 +0000 UTC m=+77.924899341 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs") pod "router-default-75bd6b5b78-gc5mg" (UID: "59387a74-b93a-49b7-86a6-4a6223fb9191") : secret "router-metrics-certs-default" not found Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.223411 2553 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:14.223535 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.223450 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls podName:a99c00fc-e6ba-450a-9505-c614036a222c nodeName:}" failed. No retries permitted until 2026-07-16 15:36:30.223443272 +0000 UTC m=+77.924995139 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-7v7l6" (UID: "a99c00fc-e6ba-450a-9505-c614036a222c") : secret "cluster-monitoring-operator-tls" not found Jul 16 15:36:14.425735 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.425708 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:14.425896 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.425842 2553 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found Jul 16 15:36:14.425941 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.425912 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls podName:18182678-feea-4569-94de-ef4be91e5a9e nodeName:}" failed. No retries permitted until 2026-07-16 15:36:30.425897216 +0000 UTC m=+78.127449087 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls") pod "cluster-samples-operator-65db858894-wg6fh" (UID: "18182678-feea-4569-94de-ef4be91e5a9e") : secret "samples-operator-tls" not found Jul 16 15:36:14.526123 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.526048 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:14.526255 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:14.526160 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:14.526255 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.526193 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:14.526255 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.526236 2553 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 16 15:36:14.526255 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.526249 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:22.526233854 +0000 UTC m=+70.227785744 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:14.526388 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:14.526283 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:22.526272231 +0000 UTC m=+70.227824099 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : configmap "kube-rbac-proxy" not found Jul 16 15:36:16.130994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:16.130969 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-jzljd" Jul 16 15:36:19.868670 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.868610 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:36:19.871710 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.871691 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 16 15:36:19.879080 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:19.879064 2553 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 16 15:36:19.879128 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:19.879119 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs podName:87f476dc-4dbe-4b63-ba23-4e6177babe08 nodeName:}" failed. No retries permitted until 2026-07-16 15:37:23.879105516 +0000 UTC m=+131.580657387 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs") pod "network-metrics-daemon-lqk7c" (UID: "87f476dc-4dbe-4b63-ba23-4e6177babe08") : secret "metrics-daemon-secret" not found Jul 16 15:36:19.882674 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.882660 2553 scope.go:117] "RemoveContainer" containerID="05d26b197e7047b76ec1ab84730ad3c51fda4c44d22b326eadd9972808b1e4fa" Jul 16 15:36:19.970074 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.970037 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:36:19.975331 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.975312 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 16 15:36:19.982991 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.982973 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/116501b0-acde-4960-b464-465ccd1d27a6-original-pull-secret\") pod \"global-pull-secret-syncer-gtpqd\" (UID: \"116501b0-acde-4960-b464-465ccd1d27a6\") " pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:36:19.998023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:19.998002 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gtpqd" Jul 16 15:36:20.071492 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.071408 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:36:20.073990 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.073966 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-flcd2\" (UniqueName: \"kubernetes.io/projected/de1997a6-b45d-4e43-aaa7-7bf05e0641e8-kube-api-access-flcd2\") pod \"network-check-target-48sll\" (UID: \"de1997a6-b45d-4e43-aaa7-7bf05e0641e8\") " pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:36:20.123609 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.123589 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gtpqd"] Jul 16 15:36:20.125867 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:20.125836 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod116501b0_acde_4960_b464_465ccd1d27a6.slice/crio-e1ef64f1744f78bb821d633bee5da7ff2b8dee4ccd9bab20c716b13d72bb8d72 WatchSource:0}: Error finding container e1ef64f1744f78bb821d633bee5da7ff2b8dee4ccd9bab20c716b13d72bb8d72: Status 404 returned error can't find the container with id e1ef64f1744f78bb821d633bee5da7ff2b8dee4ccd9bab20c716b13d72bb8d72 Jul 16 15:36:20.251671 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.251652 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:36:20.251785 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.251732 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7866497494-d4qkh" event={"ID":"c5a56410-8aa7-414a-b2ac-3614a129bf35","Type":"ContainerStarted","Data":"b0a1e933fc3da2fb16f21a8d0ba98c5dddb8e6ade4f0ae1367991f817a72fdd0"} Jul 16 15:36:20.252051 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.252028 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:36:20.253031 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.253002 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gtpqd" event={"ID":"116501b0-acde-4960-b464-465ccd1d27a6","Type":"ContainerStarted","Data":"e1ef64f1744f78bb821d633bee5da7ff2b8dee4ccd9bab20c716b13d72bb8d72"} Jul 16 15:36:20.309888 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.309868 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-2scpx\"" Jul 16 15:36:20.316505 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.316487 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:36:20.366200 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.366172 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7866497494-d4qkh" Jul 16 15:36:20.385374 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.385295 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7866497494-d4qkh" podStartSLOduration=15.874588279 podStartE2EDuration="22.38527973s" totalImagesPullingTime="6.510691451s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:20.273604991 +0000 UTC m=+67.975156893" watchObservedRunningTime="2026-07-16 15:36:20.38527973 +0000 UTC m=+68.086831625" Jul 16 15:36:20.453024 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:20.453001 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-48sll"] Jul 16 15:36:20.454876 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:20.454853 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde1997a6_b45d_4e43_aaa7_7bf05e0641e8.slice/crio-1c616b52e02797ac548c647d3756fe94729847a496ef97b1bb84d0165d82bc33 WatchSource:0}: Error finding container 1c616b52e02797ac548c647d3756fe94729847a496ef97b1bb84d0165d82bc33: Status 404 returned error can't find the container with id 1c616b52e02797ac548c647d3756fe94729847a496ef97b1bb84d0165d82bc33 Jul 16 15:36:21.258274 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:21.258241 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-48sll" event={"ID":"de1997a6-b45d-4e43-aaa7-7bf05e0641e8","Type":"ContainerStarted","Data":"fc068042ebde911df933f4005beab0a4be4d277dd2e1eb29723b3271876c1d1a"} Jul 16 15:36:21.258722 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:21.258291 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-48sll" event={"ID":"de1997a6-b45d-4e43-aaa7-7bf05e0641e8","Type":"ContainerStarted","Data":"1c616b52e02797ac548c647d3756fe94729847a496ef97b1bb84d0165d82bc33"} Jul 16 15:36:21.258722 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:21.258332 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:36:21.280442 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:21.280386 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-48sll" podStartSLOduration=66.280368093 podStartE2EDuration="1m6.280368093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:21.280257231 +0000 UTC m=+68.981809112" watchObservedRunningTime="2026-07-16 15:36:21.280368093 +0000 UTC m=+68.981919984" Jul 16 15:36:22.596541 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.596505 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:22.597023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.596553 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:22.597023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.596588 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:22.597023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.596637 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:22.597023 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:22.596722 2553 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 16 15:36:22.597023 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:22.596798 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls podName:2b974907-cb45-4089-be70-974082496bba nodeName:}" failed. No retries permitted until 2026-07-16 15:36:38.596778839 +0000 UTC m=+86.298330729 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls") pod "insights-runtime-extractor-fpmjg" (UID: "2b974907-cb45-4089-be70-974082496bba") : secret "insights-runtime-extractor-tls" not found Jul 16 15:36:22.597307 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.597187 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/2b974907-cb45-4089-be70-974082496bba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:22.599190 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.599168 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/29e3315a-0858-414b-9521-47caafd81e3a-metrics-tls\") pod \"dns-default-4qgdz\" (UID: \"29e3315a-0858-414b-9521-47caafd81e3a\") " pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:22.600276 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.600256 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"image-registry-9fcff7dcb-gsz5j\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:22.775614 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.775581 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8gcr9\"" Jul 16 15:36:22.783766 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.783738 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:22.788553 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.788533 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-vskms\"" Jul 16 15:36:22.796946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.796923 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:22.949757 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.949731 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:36:22.951246 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:22.951214 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0260ce11_3482_4709_9e13_ba6c8635c106.slice/crio-775b96a6c4b191a0deb0e5e7594a5ac60f1c16b8c3c9c7ae6acb5793bb0621e3 WatchSource:0}: Error finding container 775b96a6c4b191a0deb0e5e7594a5ac60f1c16b8c3c9c7ae6acb5793bb0621e3: Status 404 returned error can't find the container with id 775b96a6c4b191a0deb0e5e7594a5ac60f1c16b8c3c9c7ae6acb5793bb0621e3 Jul 16 15:36:22.961782 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:22.961754 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-4qgdz"] Jul 16 15:36:22.964513 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:22.964489 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29e3315a_0858_414b_9521_47caafd81e3a.slice/crio-47e41e663fbb2ae6464c186c51b44eb00dd6cb842f7b8f05f38f8b1fbe5b805f WatchSource:0}: Error finding container 47e41e663fbb2ae6464c186c51b44eb00dd6cb842f7b8f05f38f8b1fbe5b805f: Status 404 returned error can't find the container with id 47e41e663fbb2ae6464c186c51b44eb00dd6cb842f7b8f05f38f8b1fbe5b805f Jul 16 15:36:23.265312 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:23.265280 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4qgdz" event={"ID":"29e3315a-0858-414b-9521-47caafd81e3a","Type":"ContainerStarted","Data":"47e41e663fbb2ae6464c186c51b44eb00dd6cb842f7b8f05f38f8b1fbe5b805f"} Jul 16 15:36:23.266723 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:23.266696 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" event={"ID":"0260ce11-3482-4709-9e13-ba6c8635c106","Type":"ContainerStarted","Data":"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14"} Jul 16 15:36:23.266847 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:23.266741 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" event={"ID":"0260ce11-3482-4709-9e13-ba6c8635c106","Type":"ContainerStarted","Data":"775b96a6c4b191a0deb0e5e7594a5ac60f1c16b8c3c9c7ae6acb5793bb0621e3"} Jul 16 15:36:23.266909 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:23.266845 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:23.289877 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:23.289825 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" podStartSLOduration=68.28981068 podStartE2EDuration="1m8.28981068s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:23.288773103 +0000 UTC m=+70.990324993" watchObservedRunningTime="2026-07-16 15:36:23.28981068 +0000 UTC m=+70.991362569" Jul 16 15:36:26.279220 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:26.279184 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gtpqd" event={"ID":"116501b0-acde-4960-b464-465ccd1d27a6","Type":"ContainerStarted","Data":"b535b83c52d8ed2c4f7a8e891dfe1fe98047eea7bc8a5c0695dbcdd9ca43c584"} Jul 16 15:36:26.298043 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:26.297982 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gtpqd" podStartSLOduration=66.023580381 podStartE2EDuration="1m11.297962686s" totalImagesPullingTime="5.274382305s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:26.297391041 +0000 UTC m=+73.998942932" watchObservedRunningTime="2026-07-16 15:36:26.297962686 +0000 UTC m=+73.999514579" Jul 16 15:36:27.283955 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:27.283920 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4qgdz" event={"ID":"29e3315a-0858-414b-9521-47caafd81e3a","Type":"ContainerStarted","Data":"23d172689e7861de9ebf3881905b9b7011e14493bc0bda663f1a0c99e5a9ade5"} Jul 16 15:36:27.284334 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:27.283974 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-4qgdz" event={"ID":"29e3315a-0858-414b-9521-47caafd81e3a","Type":"ContainerStarted","Data":"5403200783c123644557c87ddead04b3f5a995f2adea39211fc0b3dc0d387113"} Jul 16 15:36:27.302088 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:27.302045 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-4qgdz" podStartSLOduration=33.443404002 podStartE2EDuration="37.302030786s" totalImagesPullingTime="3.858626784s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:27.299859057 +0000 UTC m=+75.001410948" watchObservedRunningTime="2026-07-16 15:36:27.302030786 +0000 UTC m=+75.003582675" Jul 16 15:36:28.287384 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:28.287351 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:30.264602 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.264563 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:30.264602 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.264636 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:30.265067 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.264670 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:30.265306 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.265286 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/59387a74-b93a-49b7-86a6-4a6223fb9191-service-ca-bundle\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:30.266988 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.266966 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/59387a74-b93a-49b7-86a6-4a6223fb9191-metrics-certs\") pod \"router-default-75bd6b5b78-gc5mg\" (UID: \"59387a74-b93a-49b7-86a6-4a6223fb9191\") " pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:30.267119 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.267036 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/a99c00fc-e6ba-450a-9505-c614036a222c-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-7v7l6\" (UID: \"a99c00fc-e6ba-450a-9505-c614036a222c\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:30.466601 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.466569 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:30.468913 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.468892 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/18182678-feea-4569-94de-ef4be91e5a9e-samples-operator-tls\") pod \"cluster-samples-operator-65db858894-wg6fh\" (UID: \"18182678-feea-4569-94de-ef4be91e5a9e\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:30.529017 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.528959 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-rnngt\"" Jul 16 15:36:30.536844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.536826 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" Jul 16 15:36:30.548770 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.548753 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-rr5xg\"" Jul 16 15:36:30.556978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.556957 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:30.674496 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.674403 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6"] Jul 16 15:36:30.676897 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:30.676870 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda99c00fc_e6ba_450a_9505_c614036a222c.slice/crio-3daa2ba973bc69103a21beee758210f0932d4284ba453708fa777924dd6d3bd3 WatchSource:0}: Error finding container 3daa2ba973bc69103a21beee758210f0932d4284ba453708fa777924dd6d3bd3: Status 404 returned error can't find the container with id 3daa2ba973bc69103a21beee758210f0932d4284ba453708fa777924dd6d3bd3 Jul 16 15:36:30.695772 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.695756 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-75bd6b5b78-gc5mg"] Jul 16 15:36:30.697991 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:30.697972 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59387a74_b93a_49b7_86a6_4a6223fb9191.slice/crio-48efb9e60e182606bb6840750724c6548b6c96d25f38460b1d27e3cb9b3a6214 WatchSource:0}: Error finding container 48efb9e60e182606bb6840750724c6548b6c96d25f38460b1d27e3cb9b3a6214: Status 404 returned error can't find the container with id 48efb9e60e182606bb6840750724c6548b6c96d25f38460b1d27e3cb9b3a6214 Jul 16 15:36:30.742927 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.742906 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-795xs\"" Jul 16 15:36:30.750545 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.750520 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" Jul 16 15:36:30.877283 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:30.877246 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh"] Jul 16 15:36:31.298969 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.298932 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" event={"ID":"59387a74-b93a-49b7-86a6-4a6223fb9191","Type":"ContainerStarted","Data":"184354ac89331109ec43c5e0a847d84ba28ae2883e1e4517243d30d27be4ecfe"} Jul 16 15:36:31.299411 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.298996 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" event={"ID":"59387a74-b93a-49b7-86a6-4a6223fb9191","Type":"ContainerStarted","Data":"48efb9e60e182606bb6840750724c6548b6c96d25f38460b1d27e3cb9b3a6214"} Jul 16 15:36:31.300289 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.300250 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" event={"ID":"18182678-feea-4569-94de-ef4be91e5a9e","Type":"ContainerStarted","Data":"aef977e5eb4a09e5bda02efb91a7259aa4ef37e19effc87706cf59b7ba513221"} Jul 16 15:36:31.301826 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.301800 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" event={"ID":"a99c00fc-e6ba-450a-9505-c614036a222c","Type":"ContainerStarted","Data":"3daa2ba973bc69103a21beee758210f0932d4284ba453708fa777924dd6d3bd3"} Jul 16 15:36:31.322172 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.322132 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" podStartSLOduration=33.322117851 podStartE2EDuration="33.322117851s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:31.321994223 +0000 UTC m=+79.023546112" watchObservedRunningTime="2026-07-16 15:36:31.322117851 +0000 UTC m=+79.023669752" Jul 16 15:36:31.558354 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.558270 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:31.561573 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.561522 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:31.595299 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.595025 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-68c9cbb497-vsj6z"] Jul 16 15:36:31.598438 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.598411 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:31.608410 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.608205 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 16 15:36:31.608956 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.608777 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-88tfc\"" Jul 16 15:36:31.609741 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.609718 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 16 15:36:31.624860 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.624805 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-vsj6z"] Jul 16 15:36:31.778518 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.778478 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xrphv\" (UniqueName: \"kubernetes.io/projected/3cc3958a-877f-4c4e-95f7-4eb756361fe2-kube-api-access-xrphv\") pod \"downloads-68c9cbb497-vsj6z\" (UID: \"3cc3958a-877f-4c4e-95f7-4eb756361fe2\") " pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:31.879823 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.879789 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xrphv\" (UniqueName: \"kubernetes.io/projected/3cc3958a-877f-4c4e-95f7-4eb756361fe2-kube-api-access-xrphv\") pod \"downloads-68c9cbb497-vsj6z\" (UID: \"3cc3958a-877f-4c4e-95f7-4eb756361fe2\") " pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:31.889554 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.889529 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xrphv\" (UniqueName: \"kubernetes.io/projected/3cc3958a-877f-4c4e-95f7-4eb756361fe2-kube-api-access-xrphv\") pod \"downloads-68c9cbb497-vsj6z\" (UID: \"3cc3958a-877f-4c4e-95f7-4eb756361fe2\") " pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:31.911071 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:31.911045 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:32.305351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:32.305275 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:32.306736 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:32.306696 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-75bd6b5b78-gc5mg" Jul 16 15:36:33.403389 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.403370 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-68c9cbb497-vsj6z"] Jul 16 15:36:33.405809 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:33.405785 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cc3958a_877f_4c4e_95f7_4eb756361fe2.slice/crio-e977b7493c84242f82f84b22007e7af4ca10807ee1fbcbd46a91b3e57f5a56b3 WatchSource:0}: Error finding container e977b7493c84242f82f84b22007e7af4ca10807ee1fbcbd46a91b3e57f5a56b3: Status 404 returned error can't find the container with id e977b7493c84242f82f84b22007e7af4ca10807ee1fbcbd46a91b3e57f5a56b3 Jul 16 15:36:33.874727 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.874694 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr"] Jul 16 15:36:33.887551 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.887524 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr"] Jul 16 15:36:33.887732 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.887615 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:33.890263 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.890219 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-stpdt\"" Jul 16 15:36:33.890829 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.890589 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 16 15:36:33.899783 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:33.899747 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-r2qrr\" (UID: \"34899729-f8b5-4a85-a55c-c5762c5fd993\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:34.000929 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.000892 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-r2qrr\" (UID: \"34899729-f8b5-4a85-a55c-c5762c5fd993\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:34.001084 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:34.001062 2553 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-admission-webhook-tls: secret "prometheus-operator-admission-webhook-tls" not found Jul 16 15:36:34.001144 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:34.001136 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates podName:34899729-f8b5-4a85-a55c-c5762c5fd993 nodeName:}" failed. No retries permitted until 2026-07-16 15:36:34.501115413 +0000 UTC m=+82.202667285 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certificates" (UniqueName: "kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates") pod "prometheus-operator-admission-webhook-77d98bc786-r2qrr" (UID: "34899729-f8b5-4a85-a55c-c5762c5fd993") : secret "prometheus-operator-admission-webhook-tls" not found Jul 16 15:36:34.316493 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.316459 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-vsj6z" event={"ID":"3cc3958a-877f-4c4e-95f7-4eb756361fe2","Type":"ContainerStarted","Data":"e977b7493c84242f82f84b22007e7af4ca10807ee1fbcbd46a91b3e57f5a56b3"} Jul 16 15:36:34.318471 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.318445 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" event={"ID":"18182678-feea-4569-94de-ef4be91e5a9e","Type":"ContainerStarted","Data":"310ec754435ecc9c62c56d8e1326347c5f69bc06b37bf946c940444b4471f056"} Jul 16 15:36:34.318609 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.318491 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" event={"ID":"18182678-feea-4569-94de-ef4be91e5a9e","Type":"ContainerStarted","Data":"b328f06c06bebd361a905abdc81811cd081febe6d5b19fbcffe172607c54a9e6"} Jul 16 15:36:34.320050 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.320022 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" event={"ID":"a99c00fc-e6ba-450a-9505-c614036a222c","Type":"ContainerStarted","Data":"b02ee2cb5aa79decc12e7cbc69100c4a53f208a1e66420be363310a7310418be"} Jul 16 15:36:34.336408 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.336365 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-65db858894-wg6fh" podStartSLOduration=33.971414943 podStartE2EDuration="36.336351186s" totalImagesPullingTime="2.364936243s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:34.335247091 +0000 UTC m=+82.036798984" watchObservedRunningTime="2026-07-16 15:36:34.336351186 +0000 UTC m=+82.037903054" Jul 16 15:36:34.352913 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.352871 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-7v7l6" podStartSLOduration=33.755120404 podStartE2EDuration="36.352859456s" totalImagesPullingTime="2.597739052s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:34.352476671 +0000 UTC m=+82.054028588" watchObservedRunningTime="2026-07-16 15:36:34.352859456 +0000 UTC m=+82.054411346" Jul 16 15:36:34.506535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.506494 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-r2qrr\" (UID: \"34899729-f8b5-4a85-a55c-c5762c5fd993\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:34.509203 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.509170 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/34899729-f8b5-4a85-a55c-c5762c5fd993-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-r2qrr\" (UID: \"34899729-f8b5-4a85-a55c-c5762c5fd993\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:34.798912 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.798876 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:34.960478 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:34.960444 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr"] Jul 16 15:36:34.963604 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:34.963570 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod34899729_f8b5_4a85_a55c_c5762c5fd993.slice/crio-ca1c5e2d8637bc20e2c780ef983778c216d7f4c74aa6f700994df3028f388837 WatchSource:0}: Error finding container ca1c5e2d8637bc20e2c780ef983778c216d7f4c74aa6f700994df3028f388837: Status 404 returned error can't find the container with id ca1c5e2d8637bc20e2c780ef983778c216d7f4c74aa6f700994df3028f388837 Jul 16 15:36:35.325327 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:35.325175 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" event={"ID":"34899729-f8b5-4a85-a55c-c5762c5fd993","Type":"ContainerStarted","Data":"ca1c5e2d8637bc20e2c780ef983778c216d7f4c74aa6f700994df3028f388837"} Jul 16 15:36:37.336413 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.336260 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" event={"ID":"34899729-f8b5-4a85-a55c-c5762c5fd993","Type":"ContainerStarted","Data":"819ba07bf85e474fae3fa7605446ffd1ec966b973469f3ea4c3e9f4a584fcd88"} Jul 16 15:36:37.337820 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.337730 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:37.339218 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.339180 2553 patch_prober.go:28] interesting pod/prometheus-operator-admission-webhook-77d98bc786-r2qrr container/prometheus-operator-admission-webhook namespace/openshift-monitoring: Readiness probe status=failure output="Get \"https://10.132.0.21:8443/healthz\": dial tcp 10.132.0.21:8443: connect: connection refused" start-of-body= Jul 16 15:36:37.339340 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.339238 2553 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" podUID="34899729-f8b5-4a85-a55c-c5762c5fd993" containerName="prometheus-operator-admission-webhook" probeResult="failure" output="Get \"https://10.132.0.21:8443/healthz\": dial tcp 10.132.0.21:8443: connect: connection refused" Jul 16 15:36:37.346870 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.346850 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:36:37.365681 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.365610 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" podStartSLOduration=2.141722872 podStartE2EDuration="4.365596182s" totalImagesPullingTime="2.22387331s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:37.36490505 +0000 UTC m=+85.066456939" watchObservedRunningTime="2026-07-16 15:36:37.365596182 +0000 UTC m=+85.067148073" Jul 16 15:36:37.366607 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.366587 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.370005 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.369982 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 16 15:36:37.370451 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.370335 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 16 15:36:37.370451 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.370358 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 16 15:36:37.370611 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.370502 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-hr47v\"" Jul 16 15:36:37.370611 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.370514 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 16 15:36:37.370853 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.370833 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:36:37.371050 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.371034 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 16 15:36:37.431129 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431095 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9zwbw\" (UniqueName: \"kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.431288 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431135 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.431288 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431208 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.431288 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.431456 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431331 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.431510 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.431466 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532718 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532630 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532718 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532683 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532718 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532711 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532835 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9zwbw\" (UniqueName: \"kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.532978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.532881 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.533437 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.533397 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.533437 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.533398 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.533643 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.533588 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.535388 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.535365 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.535767 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.535744 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.541739 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.541718 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9zwbw\" (UniqueName: \"kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw\") pod \"console-549f689bb5-bjx66\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.679454 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.679418 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:37.822216 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:37.822142 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:36:37.825141 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:37.825110 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod20350b2b_b288_4a56_b0b5_607e704bf091.slice/crio-00bb36b4296da3ec9e2d5fbf9badb722b4e3566b6c18043f71d09e4f1d47dedb WatchSource:0}: Error finding container 00bb36b4296da3ec9e2d5fbf9badb722b4e3566b6c18043f71d09e4f1d47dedb: Status 404 returned error can't find the container with id 00bb36b4296da3ec9e2d5fbf9badb722b4e3566b6c18043f71d09e4f1d47dedb Jul 16 15:36:38.292910 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.292884 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-4qgdz" Jul 16 15:36:38.342934 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.342895 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549f689bb5-bjx66" event={"ID":"20350b2b-b288-4a56-b0b5-607e704bf091","Type":"ContainerStarted","Data":"00bb36b4296da3ec9e2d5fbf9badb722b4e3566b6c18043f71d09e4f1d47dedb"} Jul 16 15:36:38.347743 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.347716 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-r2qrr" Jul 16 15:36:38.641573 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.641537 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:38.644187 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.644159 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/2b974907-cb45-4089-be70-974082496bba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-fpmjg\" (UID: \"2b974907-cb45-4089-be70-974082496bba\") " pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:38.904167 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.904088 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zzsd4\"" Jul 16 15:36:38.911669 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:38.911610 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-fpmjg" Jul 16 15:36:39.064595 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:39.064528 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-fpmjg"] Jul 16 15:36:39.068686 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:39.068658 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b974907_cb45_4089_be70_974082496bba.slice/crio-0c05f8d98273231074a71b70a5cf6383f0ef022a8be764c830c310a6b85f509f WatchSource:0}: Error finding container 0c05f8d98273231074a71b70a5cf6383f0ef022a8be764c830c310a6b85f509f: Status 404 returned error can't find the container with id 0c05f8d98273231074a71b70a5cf6383f0ef022a8be764c830c310a6b85f509f Jul 16 15:36:39.349558 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:39.349475 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpmjg" event={"ID":"2b974907-cb45-4089-be70-974082496bba","Type":"ContainerStarted","Data":"0c05f8d98273231074a71b70a5cf6383f0ef022a8be764c830c310a6b85f509f"} Jul 16 15:36:42.790484 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:42.790452 2553 patch_prober.go:28] interesting pod/image-registry-9fcff7dcb-gsz5j container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 16 15:36:42.790968 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:42.790511 2553 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 16 15:36:44.275589 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:44.275562 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:36:45.336424 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.336387 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-hrf7q"] Jul 16 15:36:45.341117 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.341093 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.343771 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.343736 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 16 15:36:45.344097 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.344071 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 16 15:36:45.344097 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.344087 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 16 15:36:45.344271 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.344067 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-4qtrk\"" Jul 16 15:36:45.344802 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.344785 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 16 15:36:45.401476 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401439 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-sys\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401720 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401483 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s94dr\" (UniqueName: \"kubernetes.io/projected/46daf924-da20-4105-9802-439becec5cfb-kube-api-access-s94dr\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401720 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401597 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-root\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401720 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401700 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-node-exporter-wtmp\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401917 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401758 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401917 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401829 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401917 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401858 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-metrics-client-ca\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.401917 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401893 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.402098 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.401938 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46daf924-da20-4105-9802-439becec5cfb-node-exporter-textfile\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502386 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502307 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-sys\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502386 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502361 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s94dr\" (UniqueName: \"kubernetes.io/projected/46daf924-da20-4105-9802-439becec5cfb-kube-api-access-s94dr\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502406 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-root\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502437 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-node-exporter-wtmp\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502447 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-sys\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502473 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502516 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-root\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502567 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-metrics-client-ca\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.502646 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502613 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.503029 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502681 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46daf924-da20-4105-9802-439becec5cfb-node-exporter-textfile\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.503029 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.502995 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/46daf924-da20-4105-9802-439becec5cfb-node-exporter-textfile\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.503122 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:45.503085 2553 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 16 15:36:45.503168 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.503142 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/46daf924-da20-4105-9802-439becec5cfb-node-exporter-wtmp\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.503168 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:36:45.503160 2553 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls podName:46daf924-da20-4105-9802-439becec5cfb nodeName:}" failed. No retries permitted until 2026-07-16 15:36:46.003139137 +0000 UTC m=+93.704691017 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls") pod "node-exporter-hrf7q" (UID: "46daf924-da20-4105-9802-439becec5cfb") : secret "node-exporter-tls" not found Jul 16 15:36:45.503508 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.503483 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-metrics-client-ca\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.503667 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.503651 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/46daf924-da20-4105-9802-439becec5cfb-node-exporter-accelerators-collector-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.505609 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.505580 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:45.515501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:45.515468 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s94dr\" (UniqueName: \"kubernetes.io/projected/46daf924-da20-4105-9802-439becec5cfb-kube-api-access-s94dr\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:46.007773 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:46.007737 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:46.010229 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:46.010209 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/46daf924-da20-4105-9802-439becec5cfb-node-exporter-tls\") pod \"node-exporter-hrf7q\" (UID: \"46daf924-da20-4105-9802-439becec5cfb\") " pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:46.252321 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:46.252284 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-hrf7q" Jul 16 15:36:50.508006 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:50.507969 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:36:52.263515 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:52.263487 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-48sll" Jul 16 15:36:52.631758 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:36:52.631725 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod46daf924_da20_4105_9802_439becec5cfb.slice/crio-65525c494b2285bea40e4800688424a4668b7c2739a060fe82f73d629f0965f7 WatchSource:0}: Error finding container 65525c494b2285bea40e4800688424a4668b7c2739a060fe82f73d629f0965f7: Status 404 returned error can't find the container with id 65525c494b2285bea40e4800688424a4668b7c2739a060fe82f73d629f0965f7 Jul 16 15:36:53.411535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.411492 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549f689bb5-bjx66" event={"ID":"20350b2b-b288-4a56-b0b5-607e704bf091","Type":"ContainerStarted","Data":"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee"} Jul 16 15:36:53.414330 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.414298 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrf7q" event={"ID":"46daf924-da20-4105-9802-439becec5cfb","Type":"ContainerStarted","Data":"65525c494b2285bea40e4800688424a4668b7c2739a060fe82f73d629f0965f7"} Jul 16 15:36:53.417903 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.417878 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpmjg" event={"ID":"2b974907-cb45-4089-be70-974082496bba","Type":"ContainerStarted","Data":"6e080fdc3714ac6264485c1183702e4f40015dfd049f58f805b0c5df30d32c53"} Jul 16 15:36:53.419969 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.419909 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-68c9cbb497-vsj6z" event={"ID":"3cc3958a-877f-4c4e-95f7-4eb756361fe2","Type":"ContainerStarted","Data":"fb06af62aa615ee2eda9212e958e64aac56133eb28e9e44d7af8f66e65722cfc"} Jul 16 15:36:53.420513 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.420487 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:53.438467 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.438234 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-549f689bb5-bjx66" podStartSLOduration=1.638500799 podStartE2EDuration="16.438216384s" totalImagesPullingTime="14.799715585s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:53.436298845 +0000 UTC m=+101.137850758" watchObservedRunningTime="2026-07-16 15:36:53.438216384 +0000 UTC m=+101.139768271" Jul 16 15:36:53.441130 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.440336 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-68c9cbb497-vsj6z" Jul 16 15:36:53.457526 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.457478 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-68c9cbb497-vsj6z" podStartSLOduration=3.191662778 podStartE2EDuration="22.457458755s" totalImagesPullingTime="19.265795977s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:36:31 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:53.45687908 +0000 UTC m=+101.158430970" watchObservedRunningTime="2026-07-16 15:36:53.457458755 +0000 UTC m=+101.159010646" Jul 16 15:36:53.607674 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:53.607602 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:36:55.433988 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:55.433947 2553 generic.go:356] "Generic (PLEG): container finished" podID="46daf924-da20-4105-9802-439becec5cfb" containerID="35398cbb3162ec601b90298b10de1a96688615361ce6597852f1ce51a1a696b0" exitCode=0 Jul 16 15:36:55.434446 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:55.434107 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrf7q" event={"ID":"46daf924-da20-4105-9802-439becec5cfb","Type":"ContainerDied","Data":"35398cbb3162ec601b90298b10de1a96688615361ce6597852f1ce51a1a696b0"} Jul 16 15:36:57.680414 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:57.680380 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:36:58.449015 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:58.448972 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrf7q" event={"ID":"46daf924-da20-4105-9802-439becec5cfb","Type":"ContainerStarted","Data":"8b22a404bd0ef4d40a671596c564c23a565483259249496175cd4cfa3fdaa119"} Jul 16 15:36:58.449185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:58.449049 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-hrf7q" event={"ID":"46daf924-da20-4105-9802-439becec5cfb","Type":"ContainerStarted","Data":"970ca6360a9e25f8a75d874d769e70ff085e692ed8169d894f7d853123909631"} Jul 16 15:36:58.450965 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:58.450940 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpmjg" event={"ID":"2b974907-cb45-4089-be70-974082496bba","Type":"ContainerStarted","Data":"9a9caab73e145ecccbf0bf7917f048f200acdc0e79b6e8083217d16f853b5853"} Jul 16 15:36:58.470475 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:36:58.470392 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-hrf7q" podStartSLOduration=11.513812841 podStartE2EDuration="13.470373772s" totalImagesPullingTime="1.859729374s" totalInitContainerRuntime="96.831557ms" isStatefulPod=true podCreationTimestamp="2026-07-16 15:36:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:36:58.469056499 +0000 UTC m=+106.170608400" watchObservedRunningTime="2026-07-16 15:36:58.470373772 +0000 UTC m=+106.171925664" Jul 16 15:37:03.475647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:03.475596 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-fpmjg" event={"ID":"2b974907-cb45-4089-be70-974082496bba","Type":"ContainerStarted","Data":"9a43ea77143f0e3ac13cd3b613aab452c364d8e5cfa45487731b4e244ed37f5f"} Jul 16 15:37:03.496285 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:03.496194 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-fpmjg" podStartSLOduration=33.523598955 podStartE2EDuration="57.4961773s" totalImagesPullingTime="23.972578345s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 15:36:06 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:37:03.495190535 +0000 UTC m=+111.196742425" watchObservedRunningTime="2026-07-16 15:37:03.4961773 +0000 UTC m=+111.197729189" Jul 16 15:37:04.628516 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.628481 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:37:04.651186 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.651154 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:37:04.651340 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.651281 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.659733 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.659707 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 16 15:37:04.777756 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777722 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.777756 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777760 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.777960 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777822 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4gvlj\" (UniqueName: \"kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.777960 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777863 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.777960 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777881 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.777960 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.777898 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.778140 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.778028 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878680 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878602 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878680 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878649 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878680 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878669 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878975 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878700 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4gvlj\" (UniqueName: \"kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878975 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878722 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878975 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878738 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.878975 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.878755 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.879453 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.879430 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.879547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.879531 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.879585 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.879540 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.879884 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.879864 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.881560 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.881538 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.881657 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.881544 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.886945 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.886927 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4gvlj\" (UniqueName: \"kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj\") pod \"console-7c94df8999-ndxgk\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:04.962219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:04.962196 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:05.116283 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:05.116261 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:37:05.118459 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:37:05.118421 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b5620a4_46ff_4da6_af7b_00d5756e29ae.slice/crio-9bb515fc13af7eee34224e46dc6a77e9c548cb38ccacb1d4f02e9d5498436862 WatchSource:0}: Error finding container 9bb515fc13af7eee34224e46dc6a77e9c548cb38ccacb1d4f02e9d5498436862: Status 404 returned error can't find the container with id 9bb515fc13af7eee34224e46dc6a77e9c548cb38ccacb1d4f02e9d5498436862 Jul 16 15:37:05.484215 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:05.484126 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94df8999-ndxgk" event={"ID":"0b5620a4-46ff-4da6-af7b-00d5756e29ae","Type":"ContainerStarted","Data":"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943"} Jul 16 15:37:05.484215 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:05.484175 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94df8999-ndxgk" event={"ID":"0b5620a4-46ff-4da6-af7b-00d5756e29ae","Type":"ContainerStarted","Data":"9bb515fc13af7eee34224e46dc6a77e9c548cb38ccacb1d4f02e9d5498436862"} Jul 16 15:37:05.505341 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:05.505293 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7c94df8999-ndxgk" podStartSLOduration=1.50527835 podStartE2EDuration="1.50527835s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:37:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:37:05.504470763 +0000 UTC m=+113.206022653" watchObservedRunningTime="2026-07-16 15:37:05.50527835 +0000 UTC m=+113.206830293" Jul 16 15:37:14.962691 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:14.962658 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:14.963273 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:14.962722 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:14.967076 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:14.967054 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:15.527078 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:15.527051 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:37:16.528042 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:16.527973 2553 generic.go:356] "Generic (PLEG): container finished" podID="2f2fe278-4fec-4892-80d4-70adfb9e3e44" containerID="082e97de54de5a20bcd6cb9db61c384c201bbc3c1d3a583ebee4330379075ced" exitCode=0 Jul 16 15:37:16.528362 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:16.528049 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" event={"ID":"2f2fe278-4fec-4892-80d4-70adfb9e3e44","Type":"ContainerDied","Data":"082e97de54de5a20bcd6cb9db61c384c201bbc3c1d3a583ebee4330379075ced"} Jul 16 15:37:16.528494 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:16.528476 2553 scope.go:117] "RemoveContainer" containerID="082e97de54de5a20bcd6cb9db61c384c201bbc3c1d3a583ebee4330379075ced" Jul 16 15:37:17.532869 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:17.532836 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-799cb7f958-npbbs" event={"ID":"2f2fe278-4fec-4892-80d4-70adfb9e3e44","Type":"ContainerStarted","Data":"eda095fbf4251bfd13791f37ab3d95ffb76b809eaea1318fce77d298b09e942b"} Jul 16 15:37:18.442147 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.442108 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-549f689bb5-bjx66" podUID="20350b2b-b288-4a56-b0b5-607e704bf091" containerName="console" containerID="cri-o://4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee" gracePeriod=15 Jul 16 15:37:18.642701 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.642638 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" containerName="registry" containerID="cri-o://81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14" gracePeriod=30 Jul 16 15:37:18.786191 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.786167 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:37:18.891665 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891634 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.891809 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891694 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9zwbw\" (UniqueName: \"kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.891809 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891762 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.891809 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891790 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.891951 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891823 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.891951 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.891861 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config\") pod \"20350b2b-b288-4a56-b0b5-607e704bf091\" (UID: \"20350b2b-b288-4a56-b0b5-607e704bf091\") " Jul 16 15:37:18.892315 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.892214 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca" (OuterVolumeSpecName: "service-ca") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:37:18.892586 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.892442 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config" (OuterVolumeSpecName: "console-config") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:37:18.892683 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.892610 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:37:18.894111 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.894081 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw" (OuterVolumeSpecName: "kube-api-access-9zwbw") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "kube-api-access-9zwbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:37:18.894234 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.894135 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:37:18.894234 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.894195 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "20350b2b-b288-4a56-b0b5-607e704bf091" (UID: "20350b2b-b288-4a56-b0b5-607e704bf091"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:37:18.930742 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.930717 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:37:18.993762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993682 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-oauth-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:18.993762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993709 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-service-ca\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:18.993762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993726 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-oauth-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:18.993762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993742 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/20350b2b-b288-4a56-b0b5-607e704bf091-console-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:18.993762 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993759 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/20350b2b-b288-4a56-b0b5-607e704bf091-console-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:18.994075 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:18.993773 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9zwbw\" (UniqueName: \"kubernetes.io/projected/20350b2b-b288-4a56-b0b5-607e704bf091-kube-api-access-9zwbw\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.094696 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094673 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.094824 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094701 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.094824 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094745 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.094891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094859 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.094927 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094895 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.094974 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094957 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.095019 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.094994 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-92jkt\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.095069 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.095049 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates\") pod \"0260ce11-3482-4709-9e13-ba6c8635c106\" (UID: \"0260ce11-3482-4709-9e13-ba6c8635c106\") " Jul 16 15:37:19.095355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.095132 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:37:19.095580 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.095500 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:37:19.097364 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.097335 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:37:19.097466 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.097440 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt" (OuterVolumeSpecName: "kube-api-access-92jkt") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "kube-api-access-92jkt". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:37:19.097466 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.097445 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:37:19.097552 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.097483 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:37:19.097552 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.097540 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:37:19.103146 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.103127 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "0260ce11-3482-4709-9e13-ba6c8635c106" (UID: "0260ce11-3482-4709-9e13-ba6c8635c106"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:37:19.196258 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196234 2553 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-image-registry-private-configuration\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196258 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196256 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-92jkt\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-kube-api-access-92jkt\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196267 2553 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-registry-certificates\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196275 2553 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-bound-sa-token\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196283 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/0260ce11-3482-4709-9e13-ba6c8635c106-trusted-ca\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196291 2553 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/0260ce11-3482-4709-9e13-ba6c8635c106-ca-trust-extracted\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196299 2553 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/0260ce11-3482-4709-9e13-ba6c8635c106-installation-pull-secrets\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.196380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.196307 2553 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/0260ce11-3482-4709-9e13-ba6c8635c106-registry-tls\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:37:19.541714 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.541686 2553 generic.go:356] "Generic (PLEG): container finished" podID="20350b2b-b288-4a56-b0b5-607e704bf091" containerID="4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee" exitCode=0 Jul 16 15:37:19.541848 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.541719 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549f689bb5-bjx66" event={"ID":"20350b2b-b288-4a56-b0b5-607e704bf091","Type":"ContainerDied","Data":"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee"} Jul 16 15:37:19.541848 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.541759 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-549f689bb5-bjx66" Jul 16 15:37:19.541848 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.541773 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-549f689bb5-bjx66" event={"ID":"20350b2b-b288-4a56-b0b5-607e704bf091","Type":"ContainerDied","Data":"00bb36b4296da3ec9e2d5fbf9badb722b4e3566b6c18043f71d09e4f1d47dedb"} Jul 16 15:37:19.541848 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.541793 2553 scope.go:117] "RemoveContainer" containerID="4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee" Jul 16 15:37:19.543079 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.543060 2553 generic.go:356] "Generic (PLEG): container finished" podID="0260ce11-3482-4709-9e13-ba6c8635c106" containerID="81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14" exitCode=0 Jul 16 15:37:19.543141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.543095 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" Jul 16 15:37:19.543141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.543106 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" event={"ID":"0260ce11-3482-4709-9e13-ba6c8635c106","Type":"ContainerDied","Data":"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14"} Jul 16 15:37:19.543219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.543145 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-9fcff7dcb-gsz5j" event={"ID":"0260ce11-3482-4709-9e13-ba6c8635c106","Type":"ContainerDied","Data":"775b96a6c4b191a0deb0e5e7594a5ac60f1c16b8c3c9c7ae6acb5793bb0621e3"} Jul 16 15:37:19.550866 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.550851 2553 scope.go:117] "RemoveContainer" containerID="4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee" Jul 16 15:37:19.551148 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:37:19.551122 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee\": container with ID starting with 4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee not found: ID does not exist" containerID="4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee" Jul 16 15:37:19.551237 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.551157 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee"} err="failed to get container status \"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee\": rpc error: code = NotFound desc = could not find container \"4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee\": container with ID starting with 4a24d8a30d131d8bf510ad127598c3c8d3116a44ffb21be3b5e677f67e4e76ee not found: ID does not exist" Jul 16 15:37:19.551237 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.551185 2553 scope.go:117] "RemoveContainer" containerID="81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14" Jul 16 15:37:19.558648 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.558634 2553 scope.go:117] "RemoveContainer" containerID="81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14" Jul 16 15:37:19.558887 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:37:19.558871 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14\": container with ID starting with 81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14 not found: ID does not exist" containerID="81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14" Jul 16 15:37:19.558973 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.558897 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14"} err="failed to get container status \"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14\": rpc error: code = NotFound desc = could not find container \"81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14\": container with ID starting with 81d8b46f6aaa09f3adf26bf1b66d7aedbf943b0bd9b9a7fd2467c094446a0d14 not found: ID does not exist" Jul 16 15:37:19.568526 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.568506 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:37:19.572121 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.572100 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-549f689bb5-bjx66"] Jul 16 15:37:19.587090 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.587069 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:37:19.594027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:19.594008 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9fcff7dcb-gsz5j"] Jul 16 15:37:20.886869 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:20.886843 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" path="/var/lib/kubelet/pods/0260ce11-3482-4709-9e13-ba6c8635c106/volumes" Jul 16 15:37:20.887341 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:20.887229 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="20350b2b-b288-4a56-b0b5-607e704bf091" path="/var/lib/kubelet/pods/20350b2b-b288-4a56-b0b5-607e704bf091/volumes" Jul 16 15:37:23.927700 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:23.927665 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:37:23.929890 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:23.929867 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/87f476dc-4dbe-4b63-ba23-4e6177babe08-metrics-certs\") pod \"network-metrics-daemon-lqk7c\" (UID: \"87f476dc-4dbe-4b63-ba23-4e6177babe08\") " pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:37:24.214130 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:24.214057 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-zfqhq\"" Jul 16 15:37:24.221401 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:24.221386 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqk7c" Jul 16 15:37:24.376569 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:24.376543 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqk7c"] Jul 16 15:37:24.378688 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:37:24.378666 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod87f476dc_4dbe_4b63_ba23_4e6177babe08.slice/crio-1db62afc96aa198974afabaae27a4e7d26b075a3f71cf65943d24aece7821d68 WatchSource:0}: Error finding container 1db62afc96aa198974afabaae27a4e7d26b075a3f71cf65943d24aece7821d68: Status 404 returned error can't find the container with id 1db62afc96aa198974afabaae27a4e7d26b075a3f71cf65943d24aece7821d68 Jul 16 15:37:24.565083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:24.565006 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqk7c" event={"ID":"87f476dc-4dbe-4b63-ba23-4e6177babe08","Type":"ContainerStarted","Data":"1db62afc96aa198974afabaae27a4e7d26b075a3f71cf65943d24aece7821d68"} Jul 16 15:37:26.574081 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:26.574057 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqk7c" event={"ID":"87f476dc-4dbe-4b63-ba23-4e6177babe08","Type":"ContainerStarted","Data":"8f19c3e9e7fc9810ad04b28e648724fb603f7f560d70c4c44d2aa7922da96f99"} Jul 16 15:37:27.579522 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:27.579484 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqk7c" event={"ID":"87f476dc-4dbe-4b63-ba23-4e6177babe08","Type":"ContainerStarted","Data":"87295b470d24e8424d583bae4744cce3fdaf5fc6185b063fec0c3b3b73ed0f70"} Jul 16 15:37:27.598084 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:27.598040 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lqk7c" podStartSLOduration=130.609367975 podStartE2EDuration="2m12.598025881s" totalImagesPullingTime="1.988657906s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:35:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:37:27.596528322 +0000 UTC m=+135.298080212" watchObservedRunningTime="2026-07-16 15:37:27.598025881 +0000 UTC m=+135.299577771" Jul 16 15:37:31.595690 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:31.595657 2553 generic.go:356] "Generic (PLEG): container finished" podID="018f2da5-c589-4c28-988f-432c9643da7c" containerID="cd09a118ac4a8b93a79e19a5fcfd48198a033c1f7a9bf0ab4176a702035d41eb" exitCode=0 Jul 16 15:37:31.596145 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:31.595715 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" event={"ID":"018f2da5-c589-4c28-988f-432c9643da7c","Type":"ContainerDied","Data":"cd09a118ac4a8b93a79e19a5fcfd48198a033c1f7a9bf0ab4176a702035d41eb"} Jul 16 15:37:31.596145 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:31.596029 2553 scope.go:117] "RemoveContainer" containerID="cd09a118ac4a8b93a79e19a5fcfd48198a033c1f7a9bf0ab4176a702035d41eb" Jul 16 15:37:32.600980 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:32.600944 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-68d9dd94cf-fzp89" event={"ID":"018f2da5-c589-4c28-988f-432c9643da7c","Type":"ContainerStarted","Data":"cd4df4e8c09cfc6ee23a1df61f5529cedd5bc1d311554d085d1c130446fe0c7f"} Jul 16 15:37:37.620822 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:37.620776 2553 generic.go:356] "Generic (PLEG): container finished" podID="bba67bb0-2159-4951-938e-cd1c782b6af4" containerID="5d82514522591940cd04e91939a363580fdb2333545882af6e25e3918d6f3d84" exitCode=0 Jul 16 15:37:37.621284 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:37.620844 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" event={"ID":"bba67bb0-2159-4951-938e-cd1c782b6af4","Type":"ContainerDied","Data":"5d82514522591940cd04e91939a363580fdb2333545882af6e25e3918d6f3d84"} Jul 16 15:37:37.621284 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:37.621221 2553 scope.go:117] "RemoveContainer" containerID="5d82514522591940cd04e91939a363580fdb2333545882af6e25e3918d6f3d84" Jul 16 15:37:38.626913 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:37:38.626882 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-7695fc6c8c-d7vd2" event={"ID":"bba67bb0-2159-4951-938e-cd1c782b6af4","Type":"ContainerStarted","Data":"0af0ffbe367ed85de582e7343f57827ca47d85b36f3ceea003ecda56d464b875"} Jul 16 15:38:07.954589 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954556 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954854 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" containerName="registry" Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954866 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" containerName="registry" Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954885 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="20350b2b-b288-4a56-b0b5-607e704bf091" containerName="console" Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954890 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="20350b2b-b288-4a56-b0b5-607e704bf091" containerName="console" Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954936 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0260ce11-3482-4709-9e13-ba6c8635c106" containerName="registry" Jul 16 15:38:07.955080 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.954948 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="20350b2b-b288-4a56-b0b5-607e704bf091" containerName="console" Jul 16 15:38:07.957969 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:07.957954 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.042831 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.042798 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.042831 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.042832 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.043052 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.042854 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.043052 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.042956 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.043052 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.042997 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nsvlw\" (UniqueName: \"kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.043190 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.043062 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.043190 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.043094 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.049522 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.049498 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:38:08.143404 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143374 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143416 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143436 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nsvlw\" (UniqueName: \"kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143463 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143480 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143502 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.143536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.143524 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.144257 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.144234 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.144355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.144284 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.144355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.144284 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.144536 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.144513 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.145997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.145977 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.146138 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.146116 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.172761 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.172730 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nsvlw\" (UniqueName: \"kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw\") pod \"console-648d9f859f-vf7cx\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.267375 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.267322 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:08.410180 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.410159 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:38:08.412103 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:38:08.412078 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e7afc67_913c_41ac_b190_895311cd5f54.slice/crio-77abe44ff16298b081394e1b9189fad91a3b1eb54f4f82b3cf0fed3962b876c6 WatchSource:0}: Error finding container 77abe44ff16298b081394e1b9189fad91a3b1eb54f4f82b3cf0fed3962b876c6: Status 404 returned error can't find the container with id 77abe44ff16298b081394e1b9189fad91a3b1eb54f4f82b3cf0fed3962b876c6 Jul 16 15:38:08.738836 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.738803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d9f859f-vf7cx" event={"ID":"0e7afc67-913c-41ac-b190-895311cd5f54","Type":"ContainerStarted","Data":"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe"} Jul 16 15:38:08.739003 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:08.738857 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d9f859f-vf7cx" event={"ID":"0e7afc67-913c-41ac-b190-895311cd5f54","Type":"ContainerStarted","Data":"77abe44ff16298b081394e1b9189fad91a3b1eb54f4f82b3cf0fed3962b876c6"} Jul 16 15:38:18.268043 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.268006 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:18.268520 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.268096 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:18.272755 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.272731 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:18.291838 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.291796 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-648d9f859f-vf7cx" podStartSLOduration=11.291783929 podStartE2EDuration="11.291783929s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:38:07 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:38:08.779750701 +0000 UTC m=+176.481302590" watchObservedRunningTime="2026-07-16 15:38:18.291783929 +0000 UTC m=+185.993335819" Jul 16 15:38:18.777948 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.777921 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:38:18.842415 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:18.842386 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:38:43.862982 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:43.862921 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7c94df8999-ndxgk" podUID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" containerName="console" containerID="cri-o://6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943" gracePeriod=15 Jul 16 15:38:44.103416 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.103395 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:38:44.208271 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208246 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208427 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208275 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208427 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208315 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208427 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208386 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208451 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208481 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208650 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208544 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4gvlj\" (UniqueName: \"kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj\") pod \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\" (UID: \"0b5620a4-46ff-4da6-af7b-00d5756e29ae\") " Jul 16 15:38:44.208712 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208684 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config" (OuterVolumeSpecName: "console-config") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:38:44.208768 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208735 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca" (OuterVolumeSpecName: "service-ca") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:38:44.208823 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208798 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:38:44.208875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208833 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.208875 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208852 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-service-ca\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.209016 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.208993 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:38:44.210564 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.210534 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:38:44.210736 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.210571 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:38:44.210736 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.210637 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj" (OuterVolumeSpecName: "kube-api-access-4gvlj") pod "0b5620a4-46ff-4da6-af7b-00d5756e29ae" (UID: "0b5620a4-46ff-4da6-af7b-00d5756e29ae"). InnerVolumeSpecName "kube-api-access-4gvlj". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:38:44.309827 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.309801 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4gvlj\" (UniqueName: \"kubernetes.io/projected/0b5620a4-46ff-4da6-af7b-00d5756e29ae-kube-api-access-4gvlj\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.309827 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.309823 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-oauth-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.309827 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.309832 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-trusted-ca-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.310002 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.309840 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0b5620a4-46ff-4da6-af7b-00d5756e29ae-console-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.310002 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.309848 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0b5620a4-46ff-4da6-af7b-00d5756e29ae-oauth-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:38:44.866485 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.866451 2553 generic.go:356] "Generic (PLEG): container finished" podID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" containerID="6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943" exitCode=0 Jul 16 15:38:44.866895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.866492 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7c94df8999-ndxgk" Jul 16 15:38:44.866895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.866535 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94df8999-ndxgk" event={"ID":"0b5620a4-46ff-4da6-af7b-00d5756e29ae","Type":"ContainerDied","Data":"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943"} Jul 16 15:38:44.866895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.866607 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7c94df8999-ndxgk" event={"ID":"0b5620a4-46ff-4da6-af7b-00d5756e29ae","Type":"ContainerDied","Data":"9bb515fc13af7eee34224e46dc6a77e9c548cb38ccacb1d4f02e9d5498436862"} Jul 16 15:38:44.866895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.866643 2553 scope.go:117] "RemoveContainer" containerID="6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943" Jul 16 15:38:44.875713 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.875696 2553 scope.go:117] "RemoveContainer" containerID="6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943" Jul 16 15:38:44.875980 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:38:44.875961 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943\": container with ID starting with 6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943 not found: ID does not exist" containerID="6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943" Jul 16 15:38:44.876027 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.875987 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943"} err="failed to get container status \"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943\": rpc error: code = NotFound desc = could not find container \"6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943\": container with ID starting with 6779233ff963344c6d33cd000e9c60f086ab01ec3cf9f330fe43a7c5a938f943 not found: ID does not exist" Jul 16 15:38:44.888645 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.888608 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:38:44.892253 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:44.892234 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7c94df8999-ndxgk"] Jul 16 15:38:46.887483 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:46.887453 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" path="/var/lib/kubelet/pods/0b5620a4-46ff-4da6-af7b-00d5756e29ae/volumes" Jul 16 15:38:56.550588 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.550555 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z"] Jul 16 15:38:56.550954 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.550866 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" containerName="console" Jul 16 15:38:56.550954 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.550878 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" containerName="console" Jul 16 15:38:56.550954 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.550938 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0b5620a4-46ff-4da6-af7b-00d5756e29ae" containerName="console" Jul 16 15:38:56.555328 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.555311 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.557982 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.557964 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 15:38:56.558059 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.557969 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mqm54\"" Jul 16 15:38:56.559009 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.558993 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 15:38:56.561835 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.561812 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z"] Jul 16 15:38:56.699970 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.699940 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9p7x\" (UniqueName: \"kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.700111 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.700005 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.700111 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.700023 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.800793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.800721 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p9p7x\" (UniqueName: \"kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.800793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.800766 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.800793 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.800785 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.801083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.801063 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.801142 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.801095 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.813300 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.813282 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9p7x\" (UniqueName: \"kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x\") pod \"685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.865122 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.865099 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:38:56.992249 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:56.992227 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z"] Jul 16 15:38:56.994240 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:38:56.994211 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod573a9670_51a2_42d9_afa5_dabc0be6262a.slice/crio-aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157 WatchSource:0}: Error finding container aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157: Status 404 returned error can't find the container with id aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157 Jul 16 15:38:57.918296 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:38:57.918252 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" event={"ID":"573a9670-51a2-42d9-afa5-dabc0be6262a","Type":"ContainerStarted","Data":"aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157"} Jul 16 15:39:03.943891 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:03.943858 2553 generic.go:356] "Generic (PLEG): container finished" podID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerID="8661eb5d6fd0ebf58161e9cfeecd36d26f88d94bcd085d1fad3f90dff8e8b27a" exitCode=0 Jul 16 15:39:03.944250 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:03.943941 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" event={"ID":"573a9670-51a2-42d9-afa5-dabc0be6262a","Type":"ContainerDied","Data":"8661eb5d6fd0ebf58161e9cfeecd36d26f88d94bcd085d1fad3f90dff8e8b27a"} Jul 16 15:39:06.957912 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:06.957883 2553 generic.go:356] "Generic (PLEG): container finished" podID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerID="db80ba3e915cff667f0a506cf15903eae996f6a32299d346e6ffa7fdf52ab437" exitCode=0 Jul 16 15:39:06.958311 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:06.957975 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" event={"ID":"573a9670-51a2-42d9-afa5-dabc0be6262a","Type":"ContainerDied","Data":"db80ba3e915cff667f0a506cf15903eae996f6a32299d346e6ffa7fdf52ab437"} Jul 16 15:39:14.993755 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:14.993720 2553 generic.go:356] "Generic (PLEG): container finished" podID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerID="b30db1779432c3e9dcf84d7fa5aa0305bb0d54353afcc082b112630bad3a0788" exitCode=0 Jul 16 15:39:14.994130 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:14.993760 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" event={"ID":"573a9670-51a2-42d9-afa5-dabc0be6262a","Type":"ContainerDied","Data":"b30db1779432c3e9dcf84d7fa5aa0305bb0d54353afcc082b112630bad3a0788"} Jul 16 15:39:16.160413 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.160387 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:39:16.245494 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.245463 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle\") pod \"573a9670-51a2-42d9-afa5-dabc0be6262a\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " Jul 16 15:39:16.245676 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.245501 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util\") pod \"573a9670-51a2-42d9-afa5-dabc0be6262a\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " Jul 16 15:39:16.245676 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.245527 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9p7x\" (UniqueName: \"kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x\") pod \"573a9670-51a2-42d9-afa5-dabc0be6262a\" (UID: \"573a9670-51a2-42d9-afa5-dabc0be6262a\") " Jul 16 15:39:16.246127 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.246100 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle" (OuterVolumeSpecName: "bundle") pod "573a9670-51a2-42d9-afa5-dabc0be6262a" (UID: "573a9670-51a2-42d9-afa5-dabc0be6262a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:16.247697 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.247673 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x" (OuterVolumeSpecName: "kube-api-access-p9p7x") pod "573a9670-51a2-42d9-afa5-dabc0be6262a" (UID: "573a9670-51a2-42d9-afa5-dabc0be6262a"). InnerVolumeSpecName "kube-api-access-p9p7x". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:39:16.250677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.250629 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util" (OuterVolumeSpecName: "util") pod "573a9670-51a2-42d9-afa5-dabc0be6262a" (UID: "573a9670-51a2-42d9-afa5-dabc0be6262a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:16.346547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.346518 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:16.346547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.346544 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/573a9670-51a2-42d9-afa5-dabc0be6262a-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:16.346731 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:16.346555 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9p7x\" (UniqueName: \"kubernetes.io/projected/573a9670-51a2-42d9-afa5-dabc0be6262a-kube-api-access-p9p7x\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:17.003036 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:17.003010 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" Jul 16 15:39:17.003182 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:17.003010 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/685f1ed643dcce69727e2826bbf9be585096b4cf774051b5daa5e67540n897z" event={"ID":"573a9670-51a2-42d9-afa5-dabc0be6262a","Type":"ContainerDied","Data":"aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157"} Jul 16 15:39:17.003182 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:17.003127 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="aa21c0fa8b62e6b636b79a8a9a26b968ed02ed646e2d1b3fafd5ce682195c157" Jul 16 15:39:24.520079 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520046 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr"] Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520320 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="util" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520330 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="util" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520341 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="extract" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520347 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="extract" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520360 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="pull" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520364 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="pull" Jul 16 15:39:24.520452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.520410 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="573a9670-51a2-42d9-afa5-dabc0be6262a" containerName="extract" Jul 16 15:39:24.523677 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.523662 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.526194 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.526175 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-677wk\"" Jul 16 15:39:24.526194 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.526177 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:39:24.526445 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.526432 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\"" Jul 16 15:39:24.534220 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.534196 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr"] Jul 16 15:39:24.607359 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.607333 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zgmx8\" (UniqueName: \"kubernetes.io/projected/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-kube-api-access-zgmx8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.607472 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.607384 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.708204 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.708177 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zgmx8\" (UniqueName: \"kubernetes.io/projected/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-kube-api-access-zgmx8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.708301 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.708232 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.708610 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.708593 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-tmp\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.717905 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.717884 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zgmx8\" (UniqueName: \"kubernetes.io/projected/bd287a3d-0dd4-48c6-94a2-272e2382a0e8-kube-api-access-zgmx8\") pod \"cert-manager-operator-controller-manager-55c444bf86-hbvvr\" (UID: \"bd287a3d-0dd4-48c6-94a2-272e2382a0e8\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.833116 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.833058 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" Jul 16 15:39:24.963979 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:24.963947 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr"] Jul 16 15:39:24.967059 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:39:24.967033 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbd287a3d_0dd4_48c6_94a2_272e2382a0e8.slice/crio-7f1ce65cb5c4ba210fd1a03ccfce4023b1f41291e48ce1efea8360b374912031 WatchSource:0}: Error finding container 7f1ce65cb5c4ba210fd1a03ccfce4023b1f41291e48ce1efea8360b374912031: Status 404 returned error can't find the container with id 7f1ce65cb5c4ba210fd1a03ccfce4023b1f41291e48ce1efea8360b374912031 Jul 16 15:39:25.034044 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:25.034018 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" event={"ID":"bd287a3d-0dd4-48c6-94a2-272e2382a0e8","Type":"ContainerStarted","Data":"7f1ce65cb5c4ba210fd1a03ccfce4023b1f41291e48ce1efea8360b374912031"} Jul 16 15:39:27.045381 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:27.045343 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" event={"ID":"bd287a3d-0dd4-48c6-94a2-272e2382a0e8","Type":"ContainerStarted","Data":"0aa202272546fe6f1a54b1a7d09f56f1b2e3c8d263f1fcc4203e40e775719b6b"} Jul 16 15:39:27.070387 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:27.070344 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-55c444bf86-hbvvr" podStartSLOduration=1.130882012 podStartE2EDuration="3.070329077s" totalImagesPullingTime="1.939447065s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:39:24 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:39:27.068493564 +0000 UTC m=+254.770045436" watchObservedRunningTime="2026-07-16 15:39:27.070329077 +0000 UTC m=+254.771881016" Jul 16 15:39:28.109290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.109261 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq"] Jul 16 15:39:28.112595 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.112576 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.116603 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.116584 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 15:39:28.116774 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.116758 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 15:39:28.117689 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.117673 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mqm54\"" Jul 16 15:39:28.125232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.125213 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq"] Jul 16 15:39:28.240948 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.240917 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msrq7\" (UniqueName: \"kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.241101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.240973 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.241101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.241048 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.341599 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.341569 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msrq7\" (UniqueName: \"kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.341794 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.341650 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.341794 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.341695 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.341987 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.341967 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.342037 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.341990 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.354230 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.354208 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msrq7\" (UniqueName: \"kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.421750 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.421727 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:28.551524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:28.551501 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq"] Jul 16 15:39:28.553563 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:39:28.553531 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podba0f03b1_20ae_48c8_bf6d_f8950dd70c83.slice/crio-f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4 WatchSource:0}: Error finding container f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4: Status 404 returned error can't find the container with id f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4 Jul 16 15:39:29.053843 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:29.053808 2553 generic.go:356] "Generic (PLEG): container finished" podID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerID="4b2c5bf2d5085ccd87fafdbb5f123f80a30ddffcd60bd1111a1ec1738eb90c13" exitCode=0 Jul 16 15:39:29.054018 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:29.053896 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" event={"ID":"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83","Type":"ContainerDied","Data":"4b2c5bf2d5085ccd87fafdbb5f123f80a30ddffcd60bd1111a1ec1738eb90c13"} Jul 16 15:39:29.054018 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:29.053983 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" event={"ID":"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83","Type":"ContainerStarted","Data":"f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4"} Jul 16 15:39:32.069411 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:32.069377 2553 generic.go:356] "Generic (PLEG): container finished" podID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerID="ebf507681d03e709dee37862b20cab5af7078158c111a77247cb50ffbcda04a2" exitCode=0 Jul 16 15:39:32.069830 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:32.069453 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" event={"ID":"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83","Type":"ContainerDied","Data":"ebf507681d03e709dee37862b20cab5af7078158c111a77247cb50ffbcda04a2"} Jul 16 15:39:33.076963 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:33.076933 2553 generic.go:356] "Generic (PLEG): container finished" podID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerID="954a108699694caad5d1507cd84308818c761a7e09b95513cb8f07052113efe1" exitCode=0 Jul 16 15:39:33.077349 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:33.077023 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" event={"ID":"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83","Type":"ContainerDied","Data":"954a108699694caad5d1507cd84308818c761a7e09b95513cb8f07052113efe1"} Jul 16 15:39:34.211280 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.211257 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:34.287707 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.287679 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msrq7\" (UniqueName: \"kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7\") pod \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " Jul 16 15:39:34.287837 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.287721 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util\") pod \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " Jul 16 15:39:34.287837 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.287756 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle\") pod \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\" (UID: \"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83\") " Jul 16 15:39:34.288161 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.288140 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle" (OuterVolumeSpecName: "bundle") pod "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" (UID: "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:34.289900 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.289867 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7" (OuterVolumeSpecName: "kube-api-access-msrq7") pod "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" (UID: "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83"). InnerVolumeSpecName "kube-api-access-msrq7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:39:34.292501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.292476 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util" (OuterVolumeSpecName: "util") pod "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" (UID: "ba0f03b1-20ae-48c8-bf6d-f8950dd70c83"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:34.389189 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.389162 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msrq7\" (UniqueName: \"kubernetes.io/projected/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-kube-api-access-msrq7\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:34.389189 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.389188 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:34.389355 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:34.389198 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ba0f03b1-20ae-48c8-bf6d-f8950dd70c83-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:35.087833 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.087805 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" Jul 16 15:39:35.088014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.087804 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87f2z6mq" event={"ID":"ba0f03b1-20ae-48c8-bf6d-f8950dd70c83","Type":"ContainerDied","Data":"f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4"} Jul 16 15:39:35.088014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.087922 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f9b8d36c4cba2b8e9d7e85e04eeff8252d1173f22d68b58044e75340be45e1a4" Jul 16 15:39:35.728332 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728299 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs"] Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728608 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="pull" Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728634 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="pull" Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728655 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="util" Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728663 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="util" Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728675 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="extract" Jul 16 15:39:35.728719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728680 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="extract" Jul 16 15:39:35.728922 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.728737 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="ba0f03b1-20ae-48c8-bf6d-f8950dd70c83" containerName="extract" Jul 16 15:39:35.731424 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.731405 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.734328 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.734300 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 16 15:39:35.735309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.735292 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-nvhdm\"" Jul 16 15:39:35.735309 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.735308 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 16 15:39:35.742835 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.742812 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs"] Jul 16 15:39:35.800889 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.800849 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.801025 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.800921 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9m7vt\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-kube-api-access-9m7vt\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.901959 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.901920 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9m7vt\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-kube-api-access-9m7vt\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.902120 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.902089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.912118 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.912087 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-bound-sa-token\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:35.912400 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:35.912376 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9m7vt\" (UniqueName: \"kubernetes.io/projected/d73d24b4-7b16-4c4f-bb4e-cdf056f26e02-kube-api-access-9m7vt\") pod \"cert-manager-cainjector-559cdd8c87-jvpqs\" (UID: \"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02\") " pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:36.059740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:36.059660 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" Jul 16 15:39:36.192933 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:36.192915 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs"] Jul 16 15:39:36.195721 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:39:36.195693 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd73d24b4_7b16_4c4f_bb4e_cdf056f26e02.slice/crio-67cb8c3c004939a895e5655798c13930338d27fc19d3635456aa4f608c37566d WatchSource:0}: Error finding container 67cb8c3c004939a895e5655798c13930338d27fc19d3635456aa4f608c37566d: Status 404 returned error can't find the container with id 67cb8c3c004939a895e5655798c13930338d27fc19d3635456aa4f608c37566d Jul 16 15:39:37.099884 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:37.099841 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" event={"ID":"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02","Type":"ContainerStarted","Data":"67cb8c3c004939a895e5655798c13930338d27fc19d3635456aa4f608c37566d"} Jul 16 15:39:38.616650 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.616602 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sq7m4"] Jul 16 15:39:38.620290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.620264 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.635452 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.635418 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sq7m4"] Jul 16 15:39:38.725568 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.725528 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5jrs\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-kube-api-access-g5jrs\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.725754 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.725588 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.826434 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.826397 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5jrs\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-kube-api-access-g5jrs\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.826590 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.826446 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.836101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.836070 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-bound-sa-token\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.838394 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.838366 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5jrs\" (UniqueName: \"kubernetes.io/projected/aa284a1c-2820-40b4-b5df-d3b2addcd0e7-kube-api-access-g5jrs\") pod \"cert-manager-55b4d954d5-sq7m4\" (UID: \"aa284a1c-2820-40b4-b5df-d3b2addcd0e7\") " pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:38.931858 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:38.931778 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-sq7m4" Jul 16 15:39:39.383421 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:39.383396 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-sq7m4"] Jul 16 15:39:39.386413 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:39:39.386388 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa284a1c_2820_40b4_b5df_d3b2addcd0e7.slice/crio-ea9b97f2d5da4b44b60bc8871c05dc6f1c0c6164ea6ffc5b5b7d6909c18c1c71 WatchSource:0}: Error finding container ea9b97f2d5da4b44b60bc8871c05dc6f1c0c6164ea6ffc5b5b7d6909c18c1c71: Status 404 returned error can't find the container with id ea9b97f2d5da4b44b60bc8871c05dc6f1c0c6164ea6ffc5b5b7d6909c18c1c71 Jul 16 15:39:40.113877 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:40.113841 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sq7m4" event={"ID":"aa284a1c-2820-40b4-b5df-d3b2addcd0e7","Type":"ContainerStarted","Data":"1151ae6eea8d3fd733c1edf301761d2fecfcc16e8cbf96f8302522fb7a6c2d5d"} Jul 16 15:39:40.113877 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:40.113890 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-sq7m4" event={"ID":"aa284a1c-2820-40b4-b5df-d3b2addcd0e7","Type":"ContainerStarted","Data":"ea9b97f2d5da4b44b60bc8871c05dc6f1c0c6164ea6ffc5b5b7d6909c18c1c71"} Jul 16 15:39:40.115269 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:40.115247 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" event={"ID":"d73d24b4-7b16-4c4f-bb4e-cdf056f26e02","Type":"ContainerStarted","Data":"7ea5f6cb3def028f394f62b1c68cca40de4894e64b12bc5e88d52aa8747e23bd"} Jul 16 15:39:40.136892 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:40.136843 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-sq7m4" podStartSLOduration=2.136828093 podStartE2EDuration="2.136828093s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:39:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:39:40.135075158 +0000 UTC m=+267.836627049" watchObservedRunningTime="2026-07-16 15:39:40.136828093 +0000 UTC m=+267.838379984" Jul 16 15:39:40.158559 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:40.158511 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-559cdd8c87-jvpqs" podStartSLOduration=2.029388503 podStartE2EDuration="5.15849653s" totalImagesPullingTime="3.129108027s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:39:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:39:40.156513126 +0000 UTC m=+267.858065015" watchObservedRunningTime="2026-07-16 15:39:40.15849653 +0000 UTC m=+267.860048419" Jul 16 15:39:48.465994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.465961 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm"] Jul 16 15:39:48.472796 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.472772 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.475871 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.475851 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 16 15:39:48.475871 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.475865 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 16 15:39:48.476986 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.476961 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-mqm54\"" Jul 16 15:39:48.479594 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.479574 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm"] Jul 16 15:39:48.603335 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.603302 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2hf7k\" (UniqueName: \"kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.603507 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.603359 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.603507 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.603420 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.704673 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.704608 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2hf7k\" (UniqueName: \"kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.704857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.704721 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.704857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.704762 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.705148 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.705123 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.705221 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.705148 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.713844 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.713816 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2hf7k\" (UniqueName: \"kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k\") pod \"629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.783230 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.783164 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:48.923239 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:48.923214 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm"] Jul 16 15:39:48.925610 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:39:48.925584 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod08a3d375_75d3_4a4d_b926_747f7d7e5570.slice/crio-f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1 WatchSource:0}: Error finding container f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1: Status 404 returned error can't find the container with id f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1 Jul 16 15:39:49.156086 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:49.156055 2553 generic.go:356] "Generic (PLEG): container finished" podID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerID="37fd48288d603a780696555fc166ad9c60cd07481481a99793157a7f97bf7559" exitCode=0 Jul 16 15:39:49.156230 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:49.156096 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" event={"ID":"08a3d375-75d3-4a4d-b926-747f7d7e5570","Type":"ContainerDied","Data":"37fd48288d603a780696555fc166ad9c60cd07481481a99793157a7f97bf7559"} Jul 16 15:39:49.156230 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:49.156138 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" event={"ID":"08a3d375-75d3-4a4d-b926-747f7d7e5570","Type":"ContainerStarted","Data":"f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1"} Jul 16 15:39:50.162857 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:50.162823 2553 generic.go:356] "Generic (PLEG): container finished" podID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerID="5bedbb4fd12eca1311b6c14ee4b59d74cce0d2f9a5722a45462f63ffa89d71cb" exitCode=0 Jul 16 15:39:50.163213 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:50.162868 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" event={"ID":"08a3d375-75d3-4a4d-b926-747f7d7e5570","Type":"ContainerDied","Data":"5bedbb4fd12eca1311b6c14ee4b59d74cce0d2f9a5722a45462f63ffa89d71cb"} Jul 16 15:39:51.168824 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:51.168789 2553 generic.go:356] "Generic (PLEG): container finished" podID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerID="c6229f37b40020d7b408dcba88f565347e52f04e2f988d4c678ebf8ab7fb2042" exitCode=0 Jul 16 15:39:51.169202 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:51.168879 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" event={"ID":"08a3d375-75d3-4a4d-b926-747f7d7e5570","Type":"ContainerDied","Data":"c6229f37b40020d7b408dcba88f565347e52f04e2f988d4c678ebf8ab7fb2042"} Jul 16 15:39:52.298134 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.298111 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:52.434813 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.434735 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2hf7k\" (UniqueName: \"kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k\") pod \"08a3d375-75d3-4a4d-b926-747f7d7e5570\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " Jul 16 15:39:52.434954 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.434814 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util\") pod \"08a3d375-75d3-4a4d-b926-747f7d7e5570\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " Jul 16 15:39:52.434954 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.434834 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle\") pod \"08a3d375-75d3-4a4d-b926-747f7d7e5570\" (UID: \"08a3d375-75d3-4a4d-b926-747f7d7e5570\") " Jul 16 15:39:52.435594 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.435566 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle" (OuterVolumeSpecName: "bundle") pod "08a3d375-75d3-4a4d-b926-747f7d7e5570" (UID: "08a3d375-75d3-4a4d-b926-747f7d7e5570"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:52.436848 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.436821 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k" (OuterVolumeSpecName: "kube-api-access-2hf7k") pod "08a3d375-75d3-4a4d-b926-747f7d7e5570" (UID: "08a3d375-75d3-4a4d-b926-747f7d7e5570"). InnerVolumeSpecName "kube-api-access-2hf7k". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:39:52.443321 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.443295 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util" (OuterVolumeSpecName: "util") pod "08a3d375-75d3-4a4d-b926-747f7d7e5570" (UID: "08a3d375-75d3-4a4d-b926-747f7d7e5570"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:39:52.535385 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.535356 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:52.535385 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.535381 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/08a3d375-75d3-4a4d-b926-747f7d7e5570-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:52.535534 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:52.535391 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2hf7k\" (UniqueName: \"kubernetes.io/projected/08a3d375-75d3-4a4d-b926-747f7d7e5570-kube-api-access-2hf7k\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:39:53.178738 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:53.178657 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" Jul 16 15:39:53.178738 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:53.178679 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/629fd74b02459b8d9ee224b8bdbb513172290d80b07a5bf327840796afj9qnm" event={"ID":"08a3d375-75d3-4a4d-b926-747f7d7e5570","Type":"ContainerDied","Data":"f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1"} Jul 16 15:39:53.178738 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:39:53.178724 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f4e2e2ae1856e3b2a504d9d7be530ec63d0061eee80afaee99f8dd3e587674b1" Jul 16 15:40:00.982468 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982426 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz"] Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982844 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="extract" Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982862 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="extract" Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982896 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="pull" Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982905 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="pull" Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982920 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="util" Jul 16 15:40:00.982946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982928 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="util" Jul 16 15:40:00.983237 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.982999 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="08a3d375-75d3-4a4d-b926-747f7d7e5570" containerName="extract" Jul 16 15:40:00.986322 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.986301 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:00.989994 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.989976 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 16 15:40:00.990093 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.990043 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-pnzkc\"" Jul 16 15:40:00.990456 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.990441 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 16 15:40:00.991160 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.991146 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 16 15:40:00.991550 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:00.991538 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 16 15:40:01.053459 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.053434 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz"] Jul 16 15:40:01.097402 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.097372 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r8hn5\" (UniqueName: \"kubernetes.io/projected/59077717-e468-4965-8e85-b8d76a7f99a4-kube-api-access-r8hn5\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.097524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.097413 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.097524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.097438 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.097524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.097477 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.097524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.097498 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198252 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-r8hn5\" (UniqueName: \"kubernetes.io/projected/59077717-e468-4965-8e85-b8d76a7f99a4-kube-api-access-r8hn5\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198295 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198505 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198322 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198505 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198349 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198505 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198377 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.198786 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.198754 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-manifests\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.200741 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.200708 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/59077717-e468-4965-8e85-b8d76a7f99a4-tmp\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.200838 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.200794 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-apiservice-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.200909 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.200892 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/59077717-e468-4965-8e85-b8d76a7f99a4-webhook-cert\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.210525 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.210492 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-r8hn5\" (UniqueName: \"kubernetes.io/projected/59077717-e468-4965-8e85-b8d76a7f99a4-kube-api-access-r8hn5\") pod \"opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz\" (UID: \"59077717-e468-4965-8e85-b8d76a7f99a4\") " pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.296511 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.296419 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:01.430238 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:01.430213 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz"] Jul 16 15:40:01.432403 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:40:01.432377 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod59077717_e468_4965_8e85_b8d76a7f99a4.slice/crio-68618cdcd0c445e76c63e063c21df725faf2b8dcda6a79e8141c469db643e189 WatchSource:0}: Error finding container 68618cdcd0c445e76c63e063c21df725faf2b8dcda6a79e8141c469db643e189: Status 404 returned error can't find the container with id 68618cdcd0c445e76c63e063c21df725faf2b8dcda6a79e8141c469db643e189 Jul 16 15:40:02.219974 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:02.219944 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" event={"ID":"59077717-e468-4965-8e85-b8d76a7f99a4","Type":"ContainerStarted","Data":"68618cdcd0c445e76c63e063c21df725faf2b8dcda6a79e8141c469db643e189"} Jul 16 15:40:06.241430 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:06.241344 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" event={"ID":"59077717-e468-4965-8e85-b8d76a7f99a4","Type":"ContainerStarted","Data":"dea844c05a9bf22c2a1a055365f55313c5a46abbca5e78d5f616c58271970770"} Jul 16 15:40:07.247245 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:07.247156 2553 generic.go:356] "Generic (PLEG): container finished" podID="59077717-e468-4965-8e85-b8d76a7f99a4" containerID="dea844c05a9bf22c2a1a055365f55313c5a46abbca5e78d5f616c58271970770" exitCode=0 Jul 16 15:40:07.247245 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:07.247203 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" event={"ID":"59077717-e468-4965-8e85-b8d76a7f99a4","Type":"ContainerDied","Data":"dea844c05a9bf22c2a1a055365f55313c5a46abbca5e78d5f616c58271970770"} Jul 16 15:40:08.254053 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:08.254014 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" event={"ID":"59077717-e468-4965-8e85-b8d76a7f99a4","Type":"ContainerStarted","Data":"aea06111e3b1bead7033a00c2e1eadefadc1eaff8abd05919047df64ecd3af01"} Jul 16 15:40:08.254437 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:08.254140 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:08.287011 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:08.286962 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" podStartSLOduration=2.2528464169999998 podStartE2EDuration="8.286946546s" totalImagesPullingTime="5.031622814s" totalInitContainerRuntime="1.002477315s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:00 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:08.28420799 +0000 UTC m=+295.985759880" watchObservedRunningTime="2026-07-16 15:40:08.286946546 +0000 UTC m=+295.988498436" Jul 16 15:40:12.732185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:12.732153 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:40:12.733829 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:12.733809 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:40:12.738271 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:12.738250 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:40:12.739930 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:12.739914 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:40:12.744280 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:12.744262 2553 kubelet.go:1648] "Image garbage collection succeeded" Jul 16 15:40:19.260341 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:19.260305 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz" Jul 16 15:40:20.364672 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.364635 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn"] Jul 16 15:40:20.370084 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.370049 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.373146 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.373123 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jul 16 15:40:20.373468 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.373453 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jul 16 15:40:20.374568 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.374545 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jul 16 15:40:20.374568 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.374560 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-6phh9\"" Jul 16 15:40:20.374758 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.374710 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jul 16 15:40:20.374819 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.374712 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jul 16 15:40:20.385089 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.385064 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn"] Jul 16 15:40:20.447683 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.447655 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bk2vx\" (UniqueName: \"kubernetes.io/projected/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-kube-api-access-bk2vx\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.447842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.447698 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.447842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.447759 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-manager-config\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.447842 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.447791 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-metrics-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.548878 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.548843 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-metrics-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.549049 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.548894 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bk2vx\" (UniqueName: \"kubernetes.io/projected/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-kube-api-access-bk2vx\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.549049 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.548966 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.549049 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.549027 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-manager-config\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.549583 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.549558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-manager-config\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.551678 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.551654 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-metrics-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.551789 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.551660 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-cert\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.557116 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.557096 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bk2vx\" (UniqueName: \"kubernetes.io/projected/2653fa9e-c76d-40aa-8469-de6e2e2e6f22-kube-api-access-bk2vx\") pod \"lws-controller-manager-75498485d8-xvjkn\" (UID: \"2653fa9e-c76d-40aa-8469-de6e2e2e6f22\") " pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.680882 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.680802 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:20.811385 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.811357 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn"] Jul 16 15:40:20.813128 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:40:20.813100 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2653fa9e_c76d_40aa_8469_de6e2e2e6f22.slice/crio-b16bb7631ab150d282bb2f5f42db68d722fe2423fe5e560ee89a3b7b14c34c4b WatchSource:0}: Error finding container b16bb7631ab150d282bb2f5f42db68d722fe2423fe5e560ee89a3b7b14c34c4b: Status 404 returned error can't find the container with id b16bb7631ab150d282bb2f5f42db68d722fe2423fe5e560ee89a3b7b14c34c4b Jul 16 15:40:20.815085 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:20.815070 2553 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 16 15:40:21.311013 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:21.310976 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" event={"ID":"2653fa9e-c76d-40aa-8469-de6e2e2e6f22","Type":"ContainerStarted","Data":"b16bb7631ab150d282bb2f5f42db68d722fe2423fe5e560ee89a3b7b14c34c4b"} Jul 16 15:40:23.322268 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:23.322235 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" event={"ID":"2653fa9e-c76d-40aa-8469-de6e2e2e6f22","Type":"ContainerStarted","Data":"058a2bf6a475e854f0d358cddc63acfbc5fc7fdd46a2261f92e5ed81ead39c34"} Jul 16 15:40:23.322758 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:23.322330 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:23.343314 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:23.343268 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" podStartSLOduration=1.8729255390000001 podStartE2EDuration="3.343252201s" totalImagesPullingTime="1.470326662s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:23.340764782 +0000 UTC m=+311.042316672" watchObservedRunningTime="2026-07-16 15:40:23.343252201 +0000 UTC m=+311.044804091" Jul 16 15:40:30.745147 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.745116 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp"] Jul 16 15:40:30.748446 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.748421 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.761029 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.761013 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-tl9p9\"" Jul 16 15:40:30.761764 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.761749 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 16 15:40:30.762293 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.762282 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 16 15:40:30.762780 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.762768 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 16 15:40:30.775000 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.774977 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 16 15:40:30.788897 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.788874 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp"] Jul 16 15:40:30.829918 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.829890 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830058 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.829929 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830058 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.829971 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9b7f04-7adc-4269-9564-b019319c30a1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830058 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.830029 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830218 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.830075 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830218 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.830140 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55n2w\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-kube-api-access-55n2w\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.830218 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.830180 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931416 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931386 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-55n2w\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-kube-api-access-55n2w\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931416 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931418 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931446 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931475 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931503 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9b7f04-7adc-4269-9564-b019319c30a1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931517 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.931647 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.931533 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.932181 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.932156 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.933941 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.933918 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.934071 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.933965 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-cacerts\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.934783 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.934759 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/de9b7f04-7adc-4269-9564-b019319c30a1-local-certs\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.934860 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.934837 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/de9b7f04-7adc-4269-9564-b019319c30a1-istio-kubeconfig\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.946642 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.946606 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-istio-token\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:30.946978 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:30.946958 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-55n2w\" (UniqueName: \"kubernetes.io/projected/de9b7f04-7adc-4269-9564-b019319c30a1-kube-api-access-55n2w\") pod \"istiod-openshift-gateway-94bb8fbfd-8pmrp\" (UID: \"de9b7f04-7adc-4269-9564-b019319c30a1\") " pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:31.057450 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:31.057396 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:31.194562 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:31.194529 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp"] Jul 16 15:40:31.197147 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:40:31.197122 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde9b7f04_7adc_4269_9564_b019319c30a1.slice/crio-3f775a316639d67c05861150b9099f3a032e5f4a86ea49a50396f1bbc49b8210 WatchSource:0}: Error finding container 3f775a316639d67c05861150b9099f3a032e5f4a86ea49a50396f1bbc49b8210: Status 404 returned error can't find the container with id 3f775a316639d67c05861150b9099f3a032e5f4a86ea49a50396f1bbc49b8210 Jul 16 15:40:31.364887 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:31.364856 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" event={"ID":"de9b7f04-7adc-4269-9564-b019319c30a1","Type":"ContainerStarted","Data":"3f775a316639d67c05861150b9099f3a032e5f4a86ea49a50396f1bbc49b8210"} Jul 16 15:40:33.523148 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:33.523100 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 15:40:33.523430 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:33.523209 2553 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"} Jul 16 15:40:34.329083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:34.329055 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-75498485d8-xvjkn" Jul 16 15:40:34.382667 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:34.382631 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" event={"ID":"de9b7f04-7adc-4269-9564-b019319c30a1","Type":"ContainerStarted","Data":"6993fde6a84da9965b40df149d919fcbbf8b700d9c242507e9159725bece8a04"} Jul 16 15:40:34.382877 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:34.382848 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:34.384429 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:34.384404 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" Jul 16 15:40:34.425152 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:40:34.425100 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-94bb8fbfd-8pmrp" podStartSLOduration=2.10128582 podStartE2EDuration="4.425081286s" totalImagesPullingTime="2.323795466s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:40:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:40:34.422803078 +0000 UTC m=+322.124354967" watchObservedRunningTime="2026-07-16 15:40:34.425081286 +0000 UTC m=+322.126633177" Jul 16 15:41:39.179237 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.179202 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5"] Jul 16 15:41:39.182661 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.182645 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.185423 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.185397 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 15:41:39.185560 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.185452 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4n4mw\"" Jul 16 15:41:39.186435 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.186415 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 15:41:39.192188 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.192166 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5"] Jul 16 15:41:39.271560 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.271530 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b84rr\" (UniqueName: \"kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.271712 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.271567 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.271712 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.271644 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.372488 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.372453 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b84rr\" (UniqueName: \"kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.372638 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.372496 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.372638 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.372531 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.372886 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.372868 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.372927 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.372897 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.380974 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.380952 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-b84rr\" (UniqueName: \"kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.491907 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.491845 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:39.625664 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.625642 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5"] Jul 16 15:41:39.627784 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:41:39.627759 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode68debf5_4acc_4f14_92e6_b07a83cfd478.slice/crio-855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0 WatchSource:0}: Error finding container 855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0: Status 404 returned error can't find the container with id 855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0 Jul 16 15:41:39.682296 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.682272 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" event={"ID":"e68debf5-4acc-4f14-92e6-b07a83cfd478","Type":"ContainerStarted","Data":"855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0"} Jul 16 15:41:39.784523 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.784458 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh"] Jul 16 15:41:39.787776 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.787761 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.799020 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.799000 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh"] Jul 16 15:41:39.878046 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.878014 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.878187 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.878051 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p5gcb\" (UniqueName: \"kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.878187 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.878083 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.978730 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.978691 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.978730 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.978731 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p5gcb\" (UniqueName: \"kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.978905 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.978752 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.979072 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.979054 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.979113 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.979087 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:39.989499 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:39.989477 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-p5gcb\" (UniqueName: \"kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:40.096038 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.095973 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:40.179718 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.179688 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4"] Jul 16 15:41:40.186845 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.186814 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.195388 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.195235 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4"] Jul 16 15:41:40.232219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.232196 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh"] Jul 16 15:41:40.233953 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:41:40.233930 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29526d22_48bd_4a0a_b8fe_326837f0827f.slice/crio-e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f WatchSource:0}: Error finding container e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f: Status 404 returned error can't find the container with id e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f Jul 16 15:41:40.281659 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.281635 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.281764 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.281674 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwjls\" (UniqueName: \"kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.281764 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.281703 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.382730 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.382703 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwjls\" (UniqueName: \"kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.382873 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.382758 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.382873 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.382844 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.383162 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.383140 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.383276 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.383159 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.397467 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.397442 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwjls\" (UniqueName: \"kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.502213 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.502189 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:40.671908 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.671885 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4"] Jul 16 15:41:40.674038 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:41:40.674013 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode8246888_9854_464c_bd31_c23354bd53fc.slice/crio-a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768 WatchSource:0}: Error finding container a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768: Status 404 returned error can't find the container with id a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768 Jul 16 15:41:40.687808 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.687783 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" event={"ID":"e8246888-9854-464c-bd31-c23354bd53fc","Type":"ContainerStarted","Data":"a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768"} Jul 16 15:41:40.689419 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.689396 2553 generic.go:356] "Generic (PLEG): container finished" podID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerID="3b55416d15f69190bfd2004e9a1feacd9b9afc73f77d691868db588b0d8c8fee" exitCode=0 Jul 16 15:41:40.689533 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.689482 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" event={"ID":"29526d22-48bd-4a0a-b8fe-326837f0827f","Type":"ContainerDied","Data":"3b55416d15f69190bfd2004e9a1feacd9b9afc73f77d691868db588b0d8c8fee"} Jul 16 15:41:40.689533 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.689523 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" event={"ID":"29526d22-48bd-4a0a-b8fe-326837f0827f","Type":"ContainerStarted","Data":"e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f"} Jul 16 15:41:40.691083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.691062 2553 generic.go:356] "Generic (PLEG): container finished" podID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerID="42b6f9a8a77209b9e57e329d3509d61f448fe382ed19b27847f9b3ea2d35525e" exitCode=0 Jul 16 15:41:40.691185 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.691102 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" event={"ID":"e68debf5-4acc-4f14-92e6-b07a83cfd478","Type":"ContainerDied","Data":"42b6f9a8a77209b9e57e329d3509d61f448fe382ed19b27847f9b3ea2d35525e"} Jul 16 15:41:40.727656 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.714004 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj"] Jul 16 15:41:40.727656 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.719555 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.753524 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.753504 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj"] Jul 16 15:41:40.785401 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.785340 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.785533 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.785445 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4dklv\" (UniqueName: \"kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.785533 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.785486 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.885885 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.885851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.886031 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.885891 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4dklv\" (UniqueName: \"kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.886031 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.885919 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.886234 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.886201 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.886294 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.886267 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:40.912253 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:40.912225 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-4dklv\" (UniqueName: \"kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:41.040235 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.040166 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:41.183893 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.183869 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj"] Jul 16 15:41:41.185635 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:41:41.185596 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde7db5a1_ece4_4ec1_bb0f_b72864d45a18.slice/crio-93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a WatchSource:0}: Error finding container 93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a: Status 404 returned error can't find the container with id 93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a Jul 16 15:41:41.697714 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.697684 2553 generic.go:356] "Generic (PLEG): container finished" podID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerID="130de81151a8a8eef628e23872078d3fa7148a2fc093469a31c53fc603f7a3b7" exitCode=0 Jul 16 15:41:41.697851 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.697760 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" event={"ID":"de7db5a1-ece4-4ec1-bb0f-b72864d45a18","Type":"ContainerDied","Data":"130de81151a8a8eef628e23872078d3fa7148a2fc093469a31c53fc603f7a3b7"} Jul 16 15:41:41.697851 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.697803 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" event={"ID":"de7db5a1-ece4-4ec1-bb0f-b72864d45a18","Type":"ContainerStarted","Data":"93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a"} Jul 16 15:41:41.699401 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.699367 2553 generic.go:356] "Generic (PLEG): container finished" podID="e8246888-9854-464c-bd31-c23354bd53fc" containerID="5ecca5a19aa8f4cb868c1478d26f62f01f2636b3103aa12f22467e291cb3e8d2" exitCode=0 Jul 16 15:41:41.699488 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.699425 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" event={"ID":"e8246888-9854-464c-bd31-c23354bd53fc","Type":"ContainerDied","Data":"5ecca5a19aa8f4cb868c1478d26f62f01f2636b3103aa12f22467e291cb3e8d2"} Jul 16 15:41:41.701425 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.701408 2553 generic.go:356] "Generic (PLEG): container finished" podID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerID="9aea37197f380f3e4203902aaa03ee96360eb85bd0d1e9d268b436dc66b39c7a" exitCode=0 Jul 16 15:41:41.701505 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:41.701431 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" event={"ID":"e68debf5-4acc-4f14-92e6-b07a83cfd478","Type":"ContainerDied","Data":"9aea37197f380f3e4203902aaa03ee96360eb85bd0d1e9d268b436dc66b39c7a"} Jul 16 15:41:42.709447 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.709422 2553 generic.go:356] "Generic (PLEG): container finished" podID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerID="99bbe883049cc9e577086b0a1dd2a148b7fecb2aef582f861d037eea8475be33" exitCode=0 Jul 16 15:41:42.709812 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.709490 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" event={"ID":"e68debf5-4acc-4f14-92e6-b07a83cfd478","Type":"ContainerDied","Data":"99bbe883049cc9e577086b0a1dd2a148b7fecb2aef582f861d037eea8475be33"} Jul 16 15:41:42.711156 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.711136 2553 generic.go:356] "Generic (PLEG): container finished" podID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerID="d767fcdb054695dc3f871ee1a50a67937f5ed6f8fbe7482821248b02196a9d21" exitCode=0 Jul 16 15:41:42.711294 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.711175 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" event={"ID":"de7db5a1-ece4-4ec1-bb0f-b72864d45a18","Type":"ContainerDied","Data":"d767fcdb054695dc3f871ee1a50a67937f5ed6f8fbe7482821248b02196a9d21"} Jul 16 15:41:42.713004 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.712984 2553 generic.go:356] "Generic (PLEG): container finished" podID="e8246888-9854-464c-bd31-c23354bd53fc" containerID="0250337c9a8f5123d7178afdd82e12684197da9bdcb3d6fda3ece97dc1f1b842" exitCode=0 Jul 16 15:41:42.713088 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:42.713067 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" event={"ID":"e8246888-9854-464c-bd31-c23354bd53fc","Type":"ContainerDied","Data":"0250337c9a8f5123d7178afdd82e12684197da9bdcb3d6fda3ece97dc1f1b842"} Jul 16 15:41:43.719456 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.719372 2553 generic.go:356] "Generic (PLEG): container finished" podID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerID="cc512ab6c698be8c18b55aff4d42443a92b510ccfa197f3e614b2df7145a46d9" exitCode=0 Jul 16 15:41:43.719456 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.719404 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" event={"ID":"de7db5a1-ece4-4ec1-bb0f-b72864d45a18","Type":"ContainerDied","Data":"cc512ab6c698be8c18b55aff4d42443a92b510ccfa197f3e614b2df7145a46d9"} Jul 16 15:41:43.721363 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.721340 2553 generic.go:356] "Generic (PLEG): container finished" podID="e8246888-9854-464c-bd31-c23354bd53fc" containerID="6b605c2cac3dd0c8f364188873ac97e437efe83cd841a544c4864d7f8b909933" exitCode=0 Jul 16 15:41:43.721484 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.721419 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" event={"ID":"e8246888-9854-464c-bd31-c23354bd53fc","Type":"ContainerDied","Data":"6b605c2cac3dd0c8f364188873ac97e437efe83cd841a544c4864d7f8b909933"} Jul 16 15:41:43.722867 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.722847 2553 generic.go:356] "Generic (PLEG): container finished" podID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerID="033ccad9b39a777186e44fd904668938b4412e9cfd0ca3e65c0324785a59494d" exitCode=0 Jul 16 15:41:43.722964 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.722925 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" event={"ID":"29526d22-48bd-4a0a-b8fe-326837f0827f","Type":"ContainerDied","Data":"033ccad9b39a777186e44fd904668938b4412e9cfd0ca3e65c0324785a59494d"} Jul 16 15:41:43.859197 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.859168 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:43.910154 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.910127 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle\") pod \"e68debf5-4acc-4f14-92e6-b07a83cfd478\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " Jul 16 15:41:43.910266 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.910173 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util\") pod \"e68debf5-4acc-4f14-92e6-b07a83cfd478\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " Jul 16 15:41:43.910325 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.910289 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b84rr\" (UniqueName: \"kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr\") pod \"e68debf5-4acc-4f14-92e6-b07a83cfd478\" (UID: \"e68debf5-4acc-4f14-92e6-b07a83cfd478\") " Jul 16 15:41:43.910901 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.910745 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle" (OuterVolumeSpecName: "bundle") pod "e68debf5-4acc-4f14-92e6-b07a83cfd478" (UID: "e68debf5-4acc-4f14-92e6-b07a83cfd478"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:43.912500 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.912475 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr" (OuterVolumeSpecName: "kube-api-access-b84rr") pod "e68debf5-4acc-4f14-92e6-b07a83cfd478" (UID: "e68debf5-4acc-4f14-92e6-b07a83cfd478"). InnerVolumeSpecName "kube-api-access-b84rr". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:41:43.915719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:43.915698 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util" (OuterVolumeSpecName: "util") pod "e68debf5-4acc-4f14-92e6-b07a83cfd478" (UID: "e68debf5-4acc-4f14-92e6-b07a83cfd478"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:44.011065 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.011009 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:44.011065 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.011031 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e68debf5-4acc-4f14-92e6-b07a83cfd478-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:44.011065 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.011040 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b84rr\" (UniqueName: \"kubernetes.io/projected/e68debf5-4acc-4f14-92e6-b07a83cfd478-kube-api-access-b84rr\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:44.729766 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.729686 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" Jul 16 15:41:44.729766 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.729713 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5" event={"ID":"e68debf5-4acc-4f14-92e6-b07a83cfd478","Type":"ContainerDied","Data":"855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0"} Jul 16 15:41:44.729766 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.729761 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="855c6bc89ac46e07bfeed34da0f8d91ddc11d88a2f1ab5ef949206e1b83747b0" Jul 16 15:41:44.731895 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.731863 2553 generic.go:356] "Generic (PLEG): container finished" podID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerID="3a3805880e3e26515063cace39c4489663bdbcf1fdb9bb750fae3ca653ac3cd1" exitCode=0 Jul 16 15:41:44.732023 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.731927 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" event={"ID":"29526d22-48bd-4a0a-b8fe-326837f0827f","Type":"ContainerDied","Data":"3a3805880e3e26515063cace39c4489663bdbcf1fdb9bb750fae3ca653ac3cd1"} Jul 16 15:41:44.868226 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.868198 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:44.890371 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.890349 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:44.917127 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917101 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vwjls\" (UniqueName: \"kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls\") pod \"e8246888-9854-464c-bd31-c23354bd53fc\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " Jul 16 15:41:44.917265 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917150 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle\") pod \"e8246888-9854-464c-bd31-c23354bd53fc\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " Jul 16 15:41:44.917265 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917193 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle\") pod \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " Jul 16 15:41:44.917265 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917259 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util\") pod \"e8246888-9854-464c-bd31-c23354bd53fc\" (UID: \"e8246888-9854-464c-bd31-c23354bd53fc\") " Jul 16 15:41:44.917530 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917297 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util\") pod \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " Jul 16 15:41:44.917530 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.917319 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4dklv\" (UniqueName: \"kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv\") pod \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\" (UID: \"de7db5a1-ece4-4ec1-bb0f-b72864d45a18\") " Jul 16 15:41:44.918112 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.918033 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle" (OuterVolumeSpecName: "bundle") pod "e8246888-9854-464c-bd31-c23354bd53fc" (UID: "e8246888-9854-464c-bd31-c23354bd53fc"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:44.920041 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.918693 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle" (OuterVolumeSpecName: "bundle") pod "de7db5a1-ece4-4ec1-bb0f-b72864d45a18" (UID: "de7db5a1-ece4-4ec1-bb0f-b72864d45a18"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:44.920510 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.920469 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls" (OuterVolumeSpecName: "kube-api-access-vwjls") pod "e8246888-9854-464c-bd31-c23354bd53fc" (UID: "e8246888-9854-464c-bd31-c23354bd53fc"). InnerVolumeSpecName "kube-api-access-vwjls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:41:44.921140 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.921115 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv" (OuterVolumeSpecName: "kube-api-access-4dklv") pod "de7db5a1-ece4-4ec1-bb0f-b72864d45a18" (UID: "de7db5a1-ece4-4ec1-bb0f-b72864d45a18"). InnerVolumeSpecName "kube-api-access-4dklv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:41:44.926820 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.926792 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util" (OuterVolumeSpecName: "util") pod "e8246888-9854-464c-bd31-c23354bd53fc" (UID: "e8246888-9854-464c-bd31-c23354bd53fc"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:44.927294 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:44.927274 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util" (OuterVolumeSpecName: "util") pod "de7db5a1-ece4-4ec1-bb0f-b72864d45a18" (UID: "de7db5a1-ece4-4ec1-bb0f-b72864d45a18"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017923 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017946 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017955 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017965 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4dklv\" (UniqueName: \"kubernetes.io/projected/de7db5a1-ece4-4ec1-bb0f-b72864d45a18-kube-api-access-4dklv\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017973 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vwjls\" (UniqueName: \"kubernetes.io/projected/e8246888-9854-464c-bd31-c23354bd53fc-kube-api-access-vwjls\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.017984 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.017982 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/e8246888-9854-464c-bd31-c23354bd53fc-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:45.738402 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.738334 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" Jul 16 15:41:45.738402 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.738359 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj" event={"ID":"de7db5a1-ece4-4ec1-bb0f-b72864d45a18","Type":"ContainerDied","Data":"93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a"} Jul 16 15:41:45.738797 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.738440 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="93aa92193b0ad68caae9abc1b7b02758c7633bd2e74b1a511280d305e97ba87a" Jul 16 15:41:45.740150 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.740134 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" Jul 16 15:41:45.740251 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.740164 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4" event={"ID":"e8246888-9854-464c-bd31-c23354bd53fc","Type":"ContainerDied","Data":"a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768"} Jul 16 15:41:45.740251 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.740198 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a832b67c5c3aebddfef207b4543c3204277b33cc5529286f49c5e7f49cb3f768" Jul 16 15:41:45.870770 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.870750 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:45.925476 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.925451 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle\") pod \"29526d22-48bd-4a0a-b8fe-326837f0827f\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " Jul 16 15:41:45.925670 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.925486 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p5gcb\" (UniqueName: \"kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb\") pod \"29526d22-48bd-4a0a-b8fe-326837f0827f\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " Jul 16 15:41:45.925670 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.925594 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util\") pod \"29526d22-48bd-4a0a-b8fe-326837f0827f\" (UID: \"29526d22-48bd-4a0a-b8fe-326837f0827f\") " Jul 16 15:41:45.926167 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.926135 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle" (OuterVolumeSpecName: "bundle") pod "29526d22-48bd-4a0a-b8fe-326837f0827f" (UID: "29526d22-48bd-4a0a-b8fe-326837f0827f"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:45.928113 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.928092 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb" (OuterVolumeSpecName: "kube-api-access-p5gcb") pod "29526d22-48bd-4a0a-b8fe-326837f0827f" (UID: "29526d22-48bd-4a0a-b8fe-326837f0827f"). InnerVolumeSpecName "kube-api-access-p5gcb". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:41:45.930935 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:45.930901 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util" (OuterVolumeSpecName: "util") pod "29526d22-48bd-4a0a-b8fe-326837f0827f" (UID: "29526d22-48bd-4a0a-b8fe-326837f0827f"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 15:41:46.026491 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.026429 2553 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-util\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:46.026491 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.026453 2553 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/29526d22-48bd-4a0a-b8fe-326837f0827f-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:46.026491 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.026462 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p5gcb\" (UniqueName: \"kubernetes.io/projected/29526d22-48bd-4a0a-b8fe-326837f0827f-kube-api-access-p5gcb\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:41:46.747060 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.747032 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" Jul 16 15:41:46.747465 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.747031 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh" event={"ID":"29526d22-48bd-4a0a-b8fe-326837f0827f","Type":"ContainerDied","Data":"e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f"} Jul 16 15:41:46.747465 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:41:46.747169 2553 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e9fcee7b9f59fe0993bdd0ff777e74bab44e19a7c1886d62924543b8c605be7f" Jul 16 15:42:25.353171 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:25.353139 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:42:50.374728 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.374638 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-648d9f859f-vf7cx" podUID="0e7afc67-913c-41ac-b190-895311cd5f54" containerName="console" containerID="cri-o://d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe" gracePeriod=15 Jul 16 15:42:50.616491 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.616465 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:42:50.618512 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618495 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nsvlw\" (UniqueName: \"kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618565 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618545 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618599 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618576 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618599 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618591 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618684 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618645 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618683 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.618719 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.618699 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert\") pod \"0e7afc67-913c-41ac-b190-895311cd5f54\" (UID: \"0e7afc67-913c-41ac-b190-895311cd5f54\") " Jul 16 15:42:50.619117 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.619008 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config" (OuterVolumeSpecName: "console-config") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:42:50.619219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.619004 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca" (OuterVolumeSpecName: "service-ca") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:42:50.619219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.619123 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:42:50.619219 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.619164 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 16 15:42:50.620729 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.620595 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw" (OuterVolumeSpecName: "kube-api-access-nsvlw") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "kube-api-access-nsvlw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 15:42:50.620729 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.620667 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:42:50.620729 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.620691 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "0e7afc67-913c-41ac-b190-895311cd5f54" (UID: "0e7afc67-913c-41ac-b190-895311cd5f54"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 15:42:50.720186 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720163 2553 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720186 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720191 2553 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-service-ca\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720203 2553 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-console-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720212 2553 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-trusted-ca-bundle\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720220 2553 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/0e7afc67-913c-41ac-b190-895311cd5f54-console-oauth-config\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720230 2553 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/0e7afc67-913c-41ac-b190-895311cd5f54-oauth-serving-cert\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:50.720351 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:50.720238 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nsvlw\" (UniqueName: \"kubernetes.io/projected/0e7afc67-913c-41ac-b190-895311cd5f54-kube-api-access-nsvlw\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 15:42:51.078380 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.078348 2553 generic.go:356] "Generic (PLEG): container finished" podID="0e7afc67-913c-41ac-b190-895311cd5f54" containerID="d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe" exitCode=0 Jul 16 15:42:51.078555 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.078397 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-648d9f859f-vf7cx" Jul 16 15:42:51.078555 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.078436 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d9f859f-vf7cx" event={"ID":"0e7afc67-913c-41ac-b190-895311cd5f54","Type":"ContainerDied","Data":"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe"} Jul 16 15:42:51.078555 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.078487 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-648d9f859f-vf7cx" event={"ID":"0e7afc67-913c-41ac-b190-895311cd5f54","Type":"ContainerDied","Data":"77abe44ff16298b081394e1b9189fad91a3b1eb54f4f82b3cf0fed3962b876c6"} Jul 16 15:42:51.078555 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.078507 2553 scope.go:117] "RemoveContainer" containerID="d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe" Jul 16 15:42:51.087202 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.087187 2553 scope.go:117] "RemoveContainer" containerID="d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe" Jul 16 15:42:51.087424 ip-10-0-134-168 kubenswrapper[2553]: E0716 15:42:51.087404 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe\": container with ID starting with d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe not found: ID does not exist" containerID="d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe" Jul 16 15:42:51.087472 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.087433 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe"} err="failed to get container status \"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe\": rpc error: code = NotFound desc = could not find container \"d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe\": container with ID starting with d742fef46cdcb6b6d3e32862b1019517fc117196059e1941be0121acb98950fe not found: ID does not exist" Jul 16 15:42:51.097386 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.097363 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:42:51.101970 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:51.101950 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-648d9f859f-vf7cx"] Jul 16 15:42:52.887927 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:42:52.887897 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0e7afc67-913c-41ac-b190-895311cd5f54" path="/var/lib/kubelet/pods/0e7afc67-913c-41ac-b190-895311cd5f54/volumes" Jul 16 15:43:06.555278 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555242 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555528 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555539 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555548 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555554 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555565 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0e7afc67-913c-41ac-b190-895311cd5f54" containerName="console" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555570 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="0e7afc67-913c-41ac-b190-895311cd5f54" containerName="console" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555576 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555581 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555589 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555594 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555599 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555604 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555616 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555637 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555651 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555659 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555673 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555681 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555688 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555694 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555700 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555705 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="extract" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555712 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555717 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="util" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555723 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="pull" Jul 16 15:43:06.555759 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555728 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="pull" Jul 16 15:43:06.556501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555779 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="0e7afc67-913c-41ac-b190-895311cd5f54" containerName="console" Jul 16 15:43:06.556501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555791 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e68debf5-4acc-4f14-92e6-b07a83cfd478" containerName="extract" Jul 16 15:43:06.556501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555796 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="29526d22-48bd-4a0a-b8fe-326837f0827f" containerName="extract" Jul 16 15:43:06.556501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555802 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="e8246888-9854-464c-bd31-c23354bd53fc" containerName="extract" Jul 16 15:43:06.556501 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.555807 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="de7db5a1-ece4-4ec1-bb0f-b72864d45a18" containerName="extract" Jul 16 15:43:06.558656 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.558616 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.561382 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.561362 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 16 15:43:06.561510 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.561413 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 16 15:43:06.562411 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.562393 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 16 15:43:06.562495 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.562412 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-4n4mw\"" Jul 16 15:43:06.567482 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.567457 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:43:06.603960 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.603934 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:43:06.643974 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.643944 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6x8wx\" (UniqueName: \"kubernetes.io/projected/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-kube-api-access-6x8wx\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.644075 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.643994 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-config-file\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.744778 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.744741 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6x8wx\" (UniqueName: \"kubernetes.io/projected/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-kube-api-access-6x8wx\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.744972 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.744822 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-config-file\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.745485 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.745455 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-config-file\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.753695 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.753664 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6x8wx\" (UniqueName: \"kubernetes.io/projected/165e6e95-37c3-42d2-ad2d-50b2fbbfe388-kube-api-access-6x8wx\") pod \"limitador-limitador-54d6d988d5-xlpt9\" (UID: \"165e6e95-37c3-42d2-ad2d-50b2fbbfe388\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:06.869782 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:06.869750 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:07.010571 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:07.010545 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:43:07.012296 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:43:07.012275 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod165e6e95_37c3_42d2_ad2d_50b2fbbfe388.slice/crio-dad628f1a1e6dd5f12c39c9d86faf2abce81d43eb6590ce14077fc266217b20f WatchSource:0}: Error finding container dad628f1a1e6dd5f12c39c9d86faf2abce81d43eb6590ce14077fc266217b20f: Status 404 returned error can't find the container with id dad628f1a1e6dd5f12c39c9d86faf2abce81d43eb6590ce14077fc266217b20f Jul 16 15:43:07.158362 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:07.158268 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" event={"ID":"165e6e95-37c3-42d2-ad2d-50b2fbbfe388","Type":"ContainerStarted","Data":"dad628f1a1e6dd5f12c39c9d86faf2abce81d43eb6590ce14077fc266217b20f"} Jul 16 15:43:10.178888 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:10.178794 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" event={"ID":"165e6e95-37c3-42d2-ad2d-50b2fbbfe388","Type":"ContainerStarted","Data":"254ecfcefc1d6a5c94450d908a9a3e4eb989aeb296eb43f7b2a6f7e8498081ee"} Jul 16 15:43:10.179246 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:10.178917 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:43:10.197950 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:10.197902 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" podStartSLOduration=1.337080801 podStartE2EDuration="4.197885865s" totalImagesPullingTime="2.860805064s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:43:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:43:10.195885724 +0000 UTC m=+477.897437627" watchObservedRunningTime="2026-07-16 15:43:10.197885865 +0000 UTC m=+477.899437787" Jul 16 15:43:21.190729 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:43:21.190688 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-xlpt9" Jul 16 15:44:21.147465 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:21.147425 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:44:37.123363 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:37.123331 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:44:42.921013 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.920977 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf"] Jul 16 15:44:42.924530 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.924513 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:42.928547 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.928525 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 16 15:44:42.928682 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.928642 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 16 15:44:42.928682 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.928643 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 16 15:44:42.928682 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.928669 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-p8txr\"" Jul 16 15:44:42.934815 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:42.934796 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf"] Jul 16 15:44:43.029979 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.029947 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02a519b2-2540-4798-aefe-ba0618933bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.029979 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.029986 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.030232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.030010 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.030232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.030037 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2jr4\" (UniqueName: \"kubernetes.io/projected/02a519b2-2540-4798-aefe-ba0618933bc4-kube-api-access-k2jr4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.030232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.030083 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.030232 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.030110 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.130953 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.130923 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02a519b2-2540-4798-aefe-ba0618933bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.130966 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.130985 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131141 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131115 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2jr4\" (UniqueName: \"kubernetes.io/projected/02a519b2-2540-4798-aefe-ba0618933bc4-kube-api-access-k2jr4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131302 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131178 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131302 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131233 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131390 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131317 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131591 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131558 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.131591 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.131587 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.133281 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.133256 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/02a519b2-2540-4798-aefe-ba0618933bc4-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.133534 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.133516 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/02a519b2-2540-4798-aefe-ba0618933bc4-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.139290 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.139273 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2jr4\" (UniqueName: \"kubernetes.io/projected/02a519b2-2540-4798-aefe-ba0618933bc4-kube-api-access-k2jr4\") pod \"e2e-distinct-simulated-kserve-59975d69f9-bs5nf\" (UID: \"02a519b2-2540-4798-aefe-ba0618933bc4\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.235105 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.235031 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:43.365846 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.365813 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf"] Jul 16 15:44:43.367444 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:44:43.367419 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02a519b2_2540_4798_aefe_ba0618933bc4.slice/crio-02c9c2288f0cfa08901ab2cd15db8e5286468cfef66f512400d217b3a3a584d8 WatchSource:0}: Error finding container 02c9c2288f0cfa08901ab2cd15db8e5286468cfef66f512400d217b3a3a584d8: Status 404 returned error can't find the container with id 02c9c2288f0cfa08901ab2cd15db8e5286468cfef66f512400d217b3a3a584d8 Jul 16 15:44:43.644691 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.644653 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" event={"ID":"02a519b2-2540-4798-aefe-ba0618933bc4","Type":"ContainerStarted","Data":"02c9c2288f0cfa08901ab2cd15db8e5286468cfef66f512400d217b3a3a584d8"} Jul 16 15:44:43.711657 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:43.711605 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:44:44.651807 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:44.651773 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" event={"ID":"02a519b2-2540-4798-aefe-ba0618933bc4","Type":"ContainerStarted","Data":"e12140405d81a563e957094d6ed11f44f318e67438dfaa14d4afa18405dd9c09"} Jul 16 15:44:44.673863 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:44.673807 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" podStartSLOduration=1.508460423 podStartE2EDuration="2.673789412s" totalImagesPullingTime="1.165328989s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:44:42 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:44:44.672200032 +0000 UTC m=+572.373751923" watchObservedRunningTime="2026-07-16 15:44:44.673789412 +0000 UTC m=+572.375341302" Jul 16 15:44:53.235740 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:53.235691 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:53.236142 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:53.235764 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:53.248353 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:53.248322 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:44:53.705459 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:44:53.705431 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-bs5nf" Jul 16 15:45:00.320997 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:00.320954 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:45:06.729343 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.729305 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl"] Jul 16 15:45:06.732907 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.732884 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.735990 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.735967 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jul 16 15:45:06.744491 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.744463 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl"] Jul 16 15:45:06.819704 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819667 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.819872 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819744 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.819872 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819793 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.819872 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819818 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.819996 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819896 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bzhv9\" (UniqueName: \"kubernetes.io/projected/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-kube-api-access-bzhv9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.819996 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.819924 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923412 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923375 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bzhv9\" (UniqueName: \"kubernetes.io/projected/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-kube-api-access-bzhv9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923423 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923479 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923505 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923553 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.923615 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923584 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.924002 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.923975 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tmp-dir\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.924085 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.924060 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.924336 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.924312 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.926238 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.926217 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.926379 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.926362 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:06.932361 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:06.932338 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bzhv9\" (UniqueName: \"kubernetes.io/projected/a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc-kube-api-access-bzhv9\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl\" (UID: \"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:07.043596 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:07.043493 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:07.181665 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:07.181639 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl"] Jul 16 15:45:07.183082 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:45:07.183056 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda47b8a1b_ba6b_4073_b7f1_25dfb8dab3cc.slice/crio-de5326ac49afa55b1dce6bc16b02ac849df354b040c0250894bb36f8948c4426 WatchSource:0}: Error finding container de5326ac49afa55b1dce6bc16b02ac849df354b040c0250894bb36f8948c4426: Status 404 returned error can't find the container with id de5326ac49afa55b1dce6bc16b02ac849df354b040c0250894bb36f8948c4426 Jul 16 15:45:07.762944 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:07.762858 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" event={"ID":"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc","Type":"ContainerStarted","Data":"f21f5df6d0f705841de03363a8cb1441df3616c3c1fe87f5127e81354b6f3467"} Jul 16 15:45:07.762944 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:07.762911 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" event={"ID":"a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc","Type":"ContainerStarted","Data":"de5326ac49afa55b1dce6bc16b02ac849df354b040c0250894bb36f8948c4426"} Jul 16 15:45:07.784629 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:07.784581 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" podStartSLOduration=1.548083629 podStartE2EDuration="1.784566285s" totalImagesPullingTime="236.482656ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:45:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:45:07.783297375 +0000 UTC m=+595.484849268" watchObservedRunningTime="2026-07-16 15:45:07.784566285 +0000 UTC m=+595.486118219" Jul 16 15:45:08.238422 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:08.238388 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:45:12.772322 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:12.772296 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:45:12.773709 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:12.773675 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:45:12.777929 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:12.777909 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:45:12.779139 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:12.779121 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:45:17.044119 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:17.044083 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:17.044503 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:17.044148 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:17.056704 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:17.056678 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:17.824031 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:17.824002 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl" Jul 16 15:45:18.218040 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.218005 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5"] Jul 16 15:45:18.221784 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.221761 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.224435 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.224418 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 16 15:45:18.233292 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.233260 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5"] Jul 16 15:45:18.320825 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.320783 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.320999 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.320900 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msmxj\" (UniqueName: \"kubernetes.io/projected/79ff5919-8b51-46d6-98e8-d6c410893337-kube-api-access-msmxj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.320999 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.320943 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.320999 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.320971 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ff5919-8b51-46d6-98e8-d6c410893337-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.320999 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.320998 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.321157 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.321020 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.421843 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421809 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-msmxj\" (UniqueName: \"kubernetes.io/projected/79ff5919-8b51-46d6-98e8-d6c410893337-kube-api-access-msmxj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422022 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421851 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422022 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421872 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ff5919-8b51-46d6-98e8-d6c410893337-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422022 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421896 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422022 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421921 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422022 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.421961 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422321 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.422299 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422373 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.422322 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.422411 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.422374 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.424101 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.424079 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/79ff5919-8b51-46d6-98e8-d6c410893337-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.424383 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.424356 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/79ff5919-8b51-46d6-98e8-d6c410893337-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.430131 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.430111 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-msmxj\" (UniqueName: \"kubernetes.io/projected/79ff5919-8b51-46d6-98e8-d6c410893337-kube-api-access-msmxj\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-5jrc5\" (UID: \"79ff5919-8b51-46d6-98e8-d6c410893337\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.531771 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.531679 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:18.663183 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.663152 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5"] Jul 16 15:45:18.663821 ip-10-0-134-168 kubenswrapper[2553]: W0716 15:45:18.663784 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod79ff5919_8b51_46d6_98e8_d6c410893337.slice/crio-8cf51dbf4f00020427de38761b3a1ea2bea481543c0ae5c007b2d78420ff68a0 WatchSource:0}: Error finding container 8cf51dbf4f00020427de38761b3a1ea2bea481543c0ae5c007b2d78420ff68a0: Status 404 returned error can't find the container with id 8cf51dbf4f00020427de38761b3a1ea2bea481543c0ae5c007b2d78420ff68a0 Jul 16 15:45:18.818000 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:18.817920 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" event={"ID":"79ff5919-8b51-46d6-98e8-d6c410893337","Type":"ContainerStarted","Data":"8cf51dbf4f00020427de38761b3a1ea2bea481543c0ae5c007b2d78420ff68a0"} Jul 16 15:45:19.824993 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:19.824954 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" event={"ID":"79ff5919-8b51-46d6-98e8-d6c410893337","Type":"ContainerStarted","Data":"b9c065b19a8892de58aa55c559fa5908bab424032cbbd3eeb15fa9768b20c933"} Jul 16 15:45:19.846379 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:19.846326 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" podStartSLOduration=1.649695451 podStartE2EDuration="1.846311286s" totalImagesPullingTime="196.615835ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 15:45:18 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 15:45:19.844262103 +0000 UTC m=+607.545813993" watchObservedRunningTime="2026-07-16 15:45:19.846311286 +0000 UTC m=+607.547863225" Jul 16 15:45:20.029435 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:20.029404 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:45:28.532870 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:28.532831 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:28.533247 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:28.532927 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:28.545202 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:28.545179 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:28.879421 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:28.879388 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-5jrc5" Jul 16 15:45:29.557950 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:45:29.557919 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:46:44.433388 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:46:44.433355 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:46:54.450438 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:46:54.450404 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:47:05.366404 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:47:05.366317 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:47:14.232972 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:47:14.232936 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:47:24.355903 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:47:24.355871 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:47:53.120546 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:47:53.120514 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:48:05.729083 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:48:05.729051 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:48:44.017815 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:48:44.017744 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:48:53.223590 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:48:53.223557 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:02.641948 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:02.641913 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:08.325049 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:08.325011 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:19.034674 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:19.034637 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:29.126738 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:29.126701 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:40.045527 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:40.045497 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:49:50.465684 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:49:50.465650 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:50:12.805034 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:12.805006 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:50:12.805374 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:12.805267 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:50:12.810638 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:12.810608 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:50:12.810824 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:12.810807 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:50:12.828946 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:12.828925 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:50:24.737808 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:24.737776 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:50:49.951469 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:50:49.951439 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:02.552581 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:02.552543 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:19.438752 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:19.438712 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:29.885606 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:29.885568 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:39.729068 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:39.728992 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:49.539731 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:49.539699 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:51:59.163105 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:51:59.163074 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:53:47.140849 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:53:47.140817 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:53:58.943862 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:53:58.943828 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:54:08.334448 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:54:08.334414 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:54:20.441976 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:54:20.441942 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:54:30.343815 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:54:30.343783 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:54:42.133223 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:54:42.133145 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:54:50.769188 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:54:50.769153 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:03.045544 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:03.045511 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:12.839014 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:12.838979 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:12.839432 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:12.839414 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:55:12.842039 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:12.842020 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 15:55:12.845701 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:12.845679 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:55:12.847929 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:12.847911 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 15:55:20.633156 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:20.633076 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:29.537484 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:29.537448 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:39.836320 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:39.836289 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:55:56.937660 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:55:56.937607 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:06.424688 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:06.424579 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:16.732307 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:16.732274 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:26.533180 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:26.533146 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:33.732469 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:33.732434 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:42.734917 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:42.734876 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:56:50.536268 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:56:50.536236 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:00.618163 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:00.618125 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:10.426845 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:10.426809 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:21.037179 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:21.037144 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:30.322187 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:30.322149 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:39.868264 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:39.868183 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:57:53.135500 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:57:53.135464 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:02.134412 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:02.134370 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:12.349424 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:12.349391 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:24.950473 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:24.950439 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:37.936448 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:37.936402 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:44.930588 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:44.930554 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:58:56.141092 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:58:56.141059 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:59:07.934106 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:59:07.934070 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:59:17.532753 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:59:17.532714 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:59:31.826446 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:59:31.826411 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:59:41.511754 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:59:41.511716 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 15:59:51.422535 ip-10-0-134-168 kubenswrapper[2553]: I0716 15:59:51.422498 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:00:09.638462 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:09.638427 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:00:12.871567 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:12.871542 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:00:12.875583 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:12.875561 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:00:12.877472 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:12.877453 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:00:12.881602 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:12.881583 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:00:21.250445 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:00:21.250410 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:02:06.927864 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:02:06.927835 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:02:14.450695 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:02:14.450658 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:02:22.616132 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:02:22.616099 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:02:34.757520 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:02:34.757478 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:02:49.145710 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:02:49.145673 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:03:30.793612 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:03:30.793577 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:03:52.029099 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:03:52.029058 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:04:15.527826 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:04:15.527781 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:04:30.342995 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:04:30.342963 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:05:09.121366 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:09.121331 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:05:12.906245 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:12.906217 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:05:12.912414 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:12.912377 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:05:12.912962 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:12.912939 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:05:12.918906 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:12.918889 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:05:18.932298 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:18.932269 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:05:30.423718 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:05:30.423686 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:06:20.124990 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:20.124954 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:06:31.129048 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:31.129011 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:06:35.204545 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.204435 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:06:35.208086 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.208068 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.211147 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.211127 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b8455d\"/\"kube-root-ca.crt\"" Jul 16 16:06:35.211241 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.211150 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-inf-a-b8455d\"/\"openshift-service-ca.crt\"" Jul 16 16:06:35.212408 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.212382 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b8455d\"/\"test-model-b8455d-kserve-self-signed-certs\"" Jul 16 16:06:35.212481 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.212419 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-inf-a-b8455d\"/\"default-dockercfg-7vngc\"" Jul 16 16:06:35.220408 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.220377 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:06:35.246708 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246676 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.246885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246715 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.246885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246748 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.246885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246826 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.246885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246860 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.247066 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.246886 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rskjf\" (UniqueName: \"kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348252 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348213 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348252 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348253 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348550 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348292 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348550 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348455 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348550 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348491 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348550 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348518 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-rskjf\" (UniqueName: \"kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348827 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348763 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348827 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348818 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.348926 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.348862 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.350695 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.350673 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.350950 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.350932 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.358480 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.358433 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-rskjf\" (UniqueName: \"kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf\") pod \"test-model-b8455d-kserve-86f64f4c65-wv2zf\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.517695 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.517583 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:35.652747 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:35.652705 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:06:35.659034 ip-10-0-134-168 kubenswrapper[2553]: W0716 16:06:35.659009 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod302f3eeb_d628_4627_84c2_becb17b053d0.slice/crio-947eacb9b98f94163bd8ad8724511da2edae0c590329da363ddc9ae3987f6cf8 WatchSource:0}: Error finding container 947eacb9b98f94163bd8ad8724511da2edae0c590329da363ddc9ae3987f6cf8: Status 404 returned error can't find the container with id 947eacb9b98f94163bd8ad8724511da2edae0c590329da363ddc9ae3987f6cf8 Jul 16 16:06:36.156444 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:36.156406 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" event={"ID":"302f3eeb-d628-4627-84c2-becb17b053d0","Type":"ContainerStarted","Data":"62e6d9de16a35523ae9c89534cf51f7f55967c89923b28ef7e60c44121da3dcf"} Jul 16 16:06:36.156673 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:36.156452 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" event={"ID":"302f3eeb-d628-4627-84c2-becb17b053d0","Type":"ContainerStarted","Data":"947eacb9b98f94163bd8ad8724511da2edae0c590329da363ddc9ae3987f6cf8"} Jul 16 16:06:36.174183 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:36.174131 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" podStartSLOduration=1.174114578 podStartE2EDuration="1.174114578s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:06:35 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:06:36.173739301 +0000 UTC m=+1883.875291192" watchObservedRunningTime="2026-07-16 16:06:36.174114578 +0000 UTC m=+1883.875666468" Jul 16 16:06:38.531689 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:38.531655 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:06:45.518441 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:45.518401 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:45.518913 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:45.518460 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:45.531069 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:45.531046 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:46.216844 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:46.216816 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:06:50.452132 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:06:50.452091 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:07:21.522890 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:21.522857 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:07:35.928714 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:35.928676 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:07:38.331358 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:38.331318 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:07:52.916300 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:52.916261 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:07:53.352831 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.352737 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" podUID="302f3eeb-d628-4627-84c2-becb17b053d0" containerName="main" containerID="cri-o://62e6d9de16a35523ae9c89534cf51f7f55967c89923b28ef7e60c44121da3dcf" gracePeriod=45 Jul 16 16:07:53.536155 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.536129 2553 generic.go:356] "Generic (PLEG): container finished" podID="302f3eeb-d628-4627-84c2-becb17b053d0" containerID="62e6d9de16a35523ae9c89534cf51f7f55967c89923b28ef7e60c44121da3dcf" exitCode=0 Jul 16 16:07:53.536289 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.536211 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" event={"ID":"302f3eeb-d628-4627-84c2-becb17b053d0","Type":"ContainerDied","Data":"62e6d9de16a35523ae9c89534cf51f7f55967c89923b28ef7e60c44121da3dcf"} Jul 16 16:07:53.607022 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.606965 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:07:53.716113 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716082 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716113 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716119 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716358 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716155 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rskjf\" (UniqueName: \"kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716358 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716222 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716358 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716272 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716358 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716340 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm\") pod \"302f3eeb-d628-4627-84c2-becb17b053d0\" (UID: \"302f3eeb-d628-4627-84c2-becb17b053d0\") " Jul 16 16:07:53.716582 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716386 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home" (OuterVolumeSpecName: "home") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:07:53.716582 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716481 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:07:53.716582 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716551 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache" (OuterVolumeSpecName: "model-cache") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:07:53.716775 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716585 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-home\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:53.716775 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.716600 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-tmp-dir\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:53.718559 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.718533 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf" (OuterVolumeSpecName: "kube-api-access-rskjf") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "kube-api-access-rskjf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:07:53.718699 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.718630 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm" (OuterVolumeSpecName: "dshm") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:07:53.718699 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.718639 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "302f3eeb-d628-4627-84c2-becb17b053d0" (UID: "302f3eeb-d628-4627-84c2-becb17b053d0"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:07:53.817782 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.817743 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/302f3eeb-d628-4627-84c2-becb17b053d0-tls-certs\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:53.817782 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.817773 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-dshm\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:53.817782 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.817783 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rskjf\" (UniqueName: \"kubernetes.io/projected/302f3eeb-d628-4627-84c2-becb17b053d0-kube-api-access-rskjf\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:53.817782 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:53.817794 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/302f3eeb-d628-4627-84c2-becb17b053d0-model-cache\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:07:54.543484 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.543455 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" Jul 16 16:07:54.543934 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.543496 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf" event={"ID":"302f3eeb-d628-4627-84c2-becb17b053d0","Type":"ContainerDied","Data":"947eacb9b98f94163bd8ad8724511da2edae0c590329da363ddc9ae3987f6cf8"} Jul 16 16:07:54.543934 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.543545 2553 scope.go:117] "RemoveContainer" containerID="62e6d9de16a35523ae9c89534cf51f7f55967c89923b28ef7e60c44121da3dcf" Jul 16 16:07:54.565718 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.565688 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:07:54.569376 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.569354 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-inf-a-b8455d/test-model-b8455d-kserve-86f64f4c65-wv2zf"] Jul 16 16:07:54.888004 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:07:54.887974 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="302f3eeb-d628-4627-84c2-becb17b053d0" path="/var/lib/kubelet/pods/302f3eeb-d628-4627-84c2-becb17b053d0/volumes" Jul 16 16:08:26.736704 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:08:26.736673 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:08:39.429847 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:08:39.429814 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:08:50.529444 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:08:50.528553 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:03.730511 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:03.730476 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:13.632855 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:13.632823 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:24.534638 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:24.534592 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:33.445245 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:33.445211 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:38.944967 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.944937 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:09:38.945396 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.945266 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="302f3eeb-d628-4627-84c2-becb17b053d0" containerName="main" Jul 16 16:09:38.945396 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.945276 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="302f3eeb-d628-4627-84c2-becb17b053d0" containerName="main" Jul 16 16:09:38.945396 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.945343 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="302f3eeb-d628-4627-84c2-becb17b053d0" containerName="main" Jul 16 16:09:38.948462 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.948447 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.955360 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.955334 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"openshift-service-ca.crt\"" Jul 16 16:09:38.955475 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.955372 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"default-dockercfg-hxqbc\"" Jul 16 16:09:38.955475 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.955344 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"auth-test-model-83e7d4-kserve-self-signed-certs\"" Jul 16 16:09:38.955642 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.955604 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-a-83e7d4\"/\"kube-root-ca.crt\"" Jul 16 16:09:38.965339 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.965317 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:09:38.971828 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.971807 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.971932 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.971843 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2c6nk\" (UniqueName: \"kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.971932 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.971877 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.972018 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.971957 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.972071 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.972054 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:38.972277 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:38.972257 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072668 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072603 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072668 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072666 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072877 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072690 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072877 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072719 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2c6nk\" (UniqueName: \"kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072877 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072751 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.072877 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.072798 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.073063 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.073040 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.073115 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.073085 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.073160 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.073132 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.074905 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.074889 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.075157 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.075140 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.083049 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.083031 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-2c6nk\" (UniqueName: \"kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk\") pod \"auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.257840 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.257749 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:39.392125 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:39.392078 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:09:39.394287 ip-10-0-134-168 kubenswrapper[2553]: W0716 16:09:39.394260 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod582f94d2_92e6_4432_9538_52a7220d4b7c.slice/crio-b16c7d0a97b28a76d3fed62350a33d4c8c6e4d26e214c4cfefce60d9ff62d7cd WatchSource:0}: Error finding container b16c7d0a97b28a76d3fed62350a33d4c8c6e4d26e214c4cfefce60d9ff62d7cd: Status 404 returned error can't find the container with id b16c7d0a97b28a76d3fed62350a33d4c8c6e4d26e214c4cfefce60d9ff62d7cd Jul 16 16:09:40.068683 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:40.068646 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" event={"ID":"582f94d2-92e6-4432-9538-52a7220d4b7c","Type":"ContainerStarted","Data":"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d"} Jul 16 16:09:40.068683 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:40.068692 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" event={"ID":"582f94d2-92e6-4432-9538-52a7220d4b7c","Type":"ContainerStarted","Data":"b16c7d0a97b28a76d3fed62350a33d4c8c6e4d26e214c4cfefce60d9ff62d7cd"} Jul 16 16:09:40.090293 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:40.090244 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" podStartSLOduration=2.090229774 podStartE2EDuration="2.090229774s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:09:38 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:09:40.088556035 +0000 UTC m=+2067.790107927" watchObservedRunningTime="2026-07-16 16:09:40.090229774 +0000 UTC m=+2067.791781664" Jul 16 16:09:43.424760 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:43.424730 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:09:49.258605 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:49.258566 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:49.259082 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:49.258638 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:49.271423 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:49.271394 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:09:50.131958 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:09:50.131927 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:10:02.636691 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:02.636042 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:10:12.939070 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:12.939038 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:10:12.944799 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:12.944778 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:10:12.946287 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:12.946267 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:10:12.952369 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:12.952350 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:10:17.945632 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:17.945598 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:10:33.827097 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:33.827066 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:10:52.235031 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:10:52.234994 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:11:10.726775 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:11:10.726735 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:11:28.339088 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:11:28.339049 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:11:47.323311 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:11:47.323270 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:12:00.877944 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:00.877905 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:12:05.150681 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:05.150639 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:12:15.899595 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:15.899535 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" podUID="582f94d2-92e6-4432-9538-52a7220d4b7c" containerName="main" containerID="cri-o://1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d" gracePeriod=45 Jul 16 16:12:16.135268 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.135244 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:12:16.280813 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280725 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2c6nk\" (UniqueName: \"kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.280813 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280770 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.280813 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280796 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.281103 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280823 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.281103 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280860 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.281103 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.280885 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home\") pod \"582f94d2-92e6-4432-9538-52a7220d4b7c\" (UID: \"582f94d2-92e6-4432-9538-52a7220d4b7c\") " Jul 16 16:12:16.281103 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.281042 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:12:16.281103 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.281079 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache" (OuterVolumeSpecName: "model-cache") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:12:16.281344 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.281222 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-tmp-dir\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.281344 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.281237 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home" (OuterVolumeSpecName: "home") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:12:16.281344 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.281241 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-model-cache\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.283070 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.283049 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk" (OuterVolumeSpecName: "kube-api-access-2c6nk") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "kube-api-access-2c6nk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:12:16.283160 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.283093 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm" (OuterVolumeSpecName: "dshm") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:12:16.283160 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.283132 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "582f94d2-92e6-4432-9538-52a7220d4b7c" (UID: "582f94d2-92e6-4432-9538-52a7220d4b7c"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:12:16.382375 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.382336 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2c6nk\" (UniqueName: \"kubernetes.io/projected/582f94d2-92e6-4432-9538-52a7220d4b7c-kube-api-access-2c6nk\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.382375 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.382368 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-dshm\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.382375 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.382381 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/582f94d2-92e6-4432-9538-52a7220d4b7c-tls-certs\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.382598 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.382392 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/582f94d2-92e6-4432-9538-52a7220d4b7c-home\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:12:16.875435 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.875396 2553 generic.go:356] "Generic (PLEG): container finished" podID="582f94d2-92e6-4432-9538-52a7220d4b7c" containerID="1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d" exitCode=0 Jul 16 16:12:16.875435 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.875435 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" Jul 16 16:12:16.875679 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.875478 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" event={"ID":"582f94d2-92e6-4432-9538-52a7220d4b7c","Type":"ContainerDied","Data":"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d"} Jul 16 16:12:16.875679 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.875520 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh" event={"ID":"582f94d2-92e6-4432-9538-52a7220d4b7c","Type":"ContainerDied","Data":"b16c7d0a97b28a76d3fed62350a33d4c8c6e4d26e214c4cfefce60d9ff62d7cd"} Jul 16 16:12:16.875679 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.875539 2553 scope.go:117] "RemoveContainer" containerID="1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d" Jul 16 16:12:16.884855 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.884837 2553 scope.go:117] "RemoveContainer" containerID="1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d" Jul 16 16:12:16.885108 ip-10-0-134-168 kubenswrapper[2553]: E0716 16:12:16.885086 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d\": container with ID starting with 1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d not found: ID does not exist" containerID="1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d" Jul 16 16:12:16.885158 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.885116 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d"} err="failed to get container status \"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d\": rpc error: code = NotFound desc = could not find container \"1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d\": container with ID starting with 1167f511a8fde6f69ca9553dbc8400393f8c4dc187cd2369a49e4be102553d1d not found: ID does not exist" Jul 16 16:12:16.926556 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.926525 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:12:16.934547 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:16.934524 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-a-83e7d4/auth-test-model-83e7d4-kserve-697db5f7dc-bq8bh"] Jul 16 16:12:18.888010 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:18.887981 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="582f94d2-92e6-4432-9538-52a7220d4b7c" path="/var/lib/kubelet/pods/582f94d2-92e6-4432-9538-52a7220d4b7c/volumes" Jul 16 16:12:24.148032 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:24.147992 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:12:53.945673 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.945587 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:12:53.946098 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.945950 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="582f94d2-92e6-4432-9538-52a7220d4b7c" containerName="main" Jul 16 16:12:53.946098 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.945963 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="582f94d2-92e6-4432-9538-52a7220d4b7c" containerName="main" Jul 16 16:12:53.946098 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.946029 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="582f94d2-92e6-4432-9538-52a7220d4b7c" containerName="main" Jul 16 16:12:53.948926 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.948909 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.952135 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.952110 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"kube-root-ca.crt\"" Jul 16 16:12:53.953482 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.953461 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"default-dockercfg-mbnkt\"" Jul 16 16:12:53.953599 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.953524 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"openshift-service-ca.crt\"" Jul 16 16:12:53.953599 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.953549 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"sub-test-model-590f9a-kserve-self-signed-certs\"" Jul 16 16:12:53.959860 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.959838 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:12:53.996123 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996097 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.996304 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996136 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.996304 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996172 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdpkk\" (UniqueName: \"kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.996304 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996280 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.996492 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996314 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:53.996492 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:53.996399 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097406 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097370 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tdpkk\" (UniqueName: \"kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097601 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097412 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097719 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097597 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097719 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097698 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097841 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097749 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097841 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097798 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.097841 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.097801 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.098058 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.098036 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.098304 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.098283 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.099695 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.099675 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.100525 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.100502 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.108450 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.107941 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdpkk\" (UniqueName: \"kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk\") pod \"sub-test-model-590f9a-kserve-64d7988c79-5xwn6\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.257827 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.257728 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:12:54.402334 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:54.402308 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:12:54.403037 ip-10-0-134-168 kubenswrapper[2553]: W0716 16:12:54.403019 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod411513ac_dfda_42db_ad71_87e070faffef.slice/crio-2c5fe5e0547beb9ae5326ed54e75e46e3b7c8a08a997bc46d7d9e6613638c64f WatchSource:0}: Error finding container 2c5fe5e0547beb9ae5326ed54e75e46e3b7c8a08a997bc46d7d9e6613638c64f: Status 404 returned error can't find the container with id 2c5fe5e0547beb9ae5326ed54e75e46e3b7c8a08a997bc46d7d9e6613638c64f Jul 16 16:12:55.069470 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:55.069428 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" event={"ID":"411513ac-dfda-42db-ad71-87e070faffef","Type":"ContainerStarted","Data":"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc"} Jul 16 16:12:55.069886 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:55.069478 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" event={"ID":"411513ac-dfda-42db-ad71-87e070faffef","Type":"ContainerStarted","Data":"2c5fe5e0547beb9ae5326ed54e75e46e3b7c8a08a997bc46d7d9e6613638c64f"} Jul 16 16:12:55.088081 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:55.088033 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" podStartSLOduration=2.088018651 podStartE2EDuration="2.088018651s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:12:53 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:12:55.08733556 +0000 UTC m=+2262.788887450" watchObservedRunningTime="2026-07-16 16:12:55.088018651 +0000 UTC m=+2262.789570540" Jul 16 16:12:59.220895 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:12:59.220859 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:13:04.258713 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:04.258672 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:04.259100 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:04.258725 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:04.271109 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:04.271081 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:05.135201 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:05.135171 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:26.326038 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:26.326001 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:13:43.052040 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:43.052008 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:13:47.468329 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:47.468292 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:13:58.071898 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.071853 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" podUID="411513ac-dfda-42db-ad71-87e070faffef" containerName="main" containerID="cri-o://07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc" gracePeriod=45 Jul 16 16:13:58.322952 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.322889 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:58.386267 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.386232 2553 generic.go:356] "Generic (PLEG): container finished" podID="411513ac-dfda-42db-ad71-87e070faffef" containerID="07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc" exitCode=0 Jul 16 16:13:58.386267 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.386268 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" Jul 16 16:13:58.386469 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.386318 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" event={"ID":"411513ac-dfda-42db-ad71-87e070faffef","Type":"ContainerDied","Data":"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc"} Jul 16 16:13:58.386469 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.386357 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6" event={"ID":"411513ac-dfda-42db-ad71-87e070faffef","Type":"ContainerDied","Data":"2c5fe5e0547beb9ae5326ed54e75e46e3b7c8a08a997bc46d7d9e6613638c64f"} Jul 16 16:13:58.386469 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.386377 2553 scope.go:117] "RemoveContainer" containerID="07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc" Jul 16 16:13:58.395038 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.395018 2553 scope.go:117] "RemoveContainer" containerID="07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc" Jul 16 16:13:58.395273 ip-10-0-134-168 kubenswrapper[2553]: E0716 16:13:58.395254 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc\": container with ID starting with 07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc not found: ID does not exist" containerID="07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc" Jul 16 16:13:58.395320 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.395285 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc"} err="failed to get container status \"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc\": rpc error: code = NotFound desc = could not find container \"07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc\": container with ID starting with 07df793792e718a9693e1dd01de14c0a91d0a2e00d97e33a4701f2bf62f4a2fc not found: ID does not exist" Jul 16 16:13:58.447746 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447719 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdpkk\" (UniqueName: \"kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.447887 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447762 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.447887 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447784 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.447887 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447852 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.447887 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447874 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.448099 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.447899 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir\") pod \"411513ac-dfda-42db-ad71-87e070faffef\" (UID: \"411513ac-dfda-42db-ad71-87e070faffef\") " Jul 16 16:13:58.448099 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.448041 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home" (OuterVolumeSpecName: "home") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:13:58.448207 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.448158 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache" (OuterVolumeSpecName: "model-cache") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:13:58.448207 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.448173 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-home\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.448294 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.448210 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:13:58.450136 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.450114 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm" (OuterVolumeSpecName: "dshm") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:13:58.450220 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.450132 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:13:58.450220 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.450148 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk" (OuterVolumeSpecName: "kube-api-access-tdpkk") pod "411513ac-dfda-42db-ad71-87e070faffef" (UID: "411513ac-dfda-42db-ad71-87e070faffef"). InnerVolumeSpecName "kube-api-access-tdpkk". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:13:58.549558 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.549522 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-dshm\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.549558 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.549549 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-model-cache\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.549558 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.549560 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/411513ac-dfda-42db-ad71-87e070faffef-tmp-dir\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.549810 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.549569 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdpkk\" (UniqueName: \"kubernetes.io/projected/411513ac-dfda-42db-ad71-87e070faffef-kube-api-access-tdpkk\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.549810 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.549577 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/411513ac-dfda-42db-ad71-87e070faffef-tls-certs\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:13:58.711313 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.711283 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:13:58.718316 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.718295 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/sub-test-model-590f9a-kserve-64d7988c79-5xwn6"] Jul 16 16:13:58.887635 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:13:58.887586 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="411513ac-dfda-42db-ad71-87e070faffef" path="/var/lib/kubelet/pods/411513ac-dfda-42db-ad71-87e070faffef/volumes" Jul 16 16:14:02.772776 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:02.772738 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:14:17.078196 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.078159 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:14:17.078558 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.078524 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="411513ac-dfda-42db-ad71-87e070faffef" containerName="main" Jul 16 16:14:17.078558 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.078536 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="411513ac-dfda-42db-ad71-87e070faffef" containerName="main" Jul 16 16:14:17.078667 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.078594 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="411513ac-dfda-42db-ad71-87e070faffef" containerName="main" Jul 16 16:14:17.082868 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.082850 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.106117 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.106096 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"kube-root-ca.crt\"" Jul 16 16:14:17.106999 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.106976 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"default-dockercfg-mbnkt\"" Jul 16 16:14:17.107114 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.106977 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"rate-test-model-590f9a-kserve-self-signed-certs\"" Jul 16 16:14:17.107114 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.107019 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"ai-tenant-e2e-shared-b-590f9a\"/\"openshift-service-ca.crt\"" Jul 16 16:14:17.131492 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.131468 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:14:17.214164 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214135 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.214308 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214174 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.214308 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214203 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.214308 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214270 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.214413 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214346 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.214413 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.214376 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cv6xv\" (UniqueName: \"kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.314764 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.314733 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.314936 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.314771 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cv6xv\" (UniqueName: \"kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.314936 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.314805 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.314936 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.314831 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.315107 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.314962 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.315107 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.315014 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.315213 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.315190 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.315259 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.315228 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.315319 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.315268 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.317162 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.317137 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.317349 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.317330 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.329418 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.329357 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cv6xv\" (UniqueName: \"kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv\") pod \"rate-test-model-590f9a-kserve-78cd9f955f-6sq77\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.391343 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.391315 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:17.529142 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:17.529117 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:14:17.530340 ip-10-0-134-168 kubenswrapper[2553]: W0716 16:14:17.530318 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod82e21c79_2ad8_498c_8af1_2efc049b12d6.slice/crio-f8d379b9ae1befef07a9b413646eaf1530fa0f922e3cf60b18da48b1c826be21 WatchSource:0}: Error finding container f8d379b9ae1befef07a9b413646eaf1530fa0f922e3cf60b18da48b1c826be21: Status 404 returned error can't find the container with id f8d379b9ae1befef07a9b413646eaf1530fa0f922e3cf60b18da48b1c826be21 Jul 16 16:14:18.494419 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:18.494386 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" event={"ID":"82e21c79-2ad8-498c-8af1-2efc049b12d6","Type":"ContainerStarted","Data":"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3"} Jul 16 16:14:18.494817 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:18.494429 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" event={"ID":"82e21c79-2ad8-498c-8af1-2efc049b12d6","Type":"ContainerStarted","Data":"f8d379b9ae1befef07a9b413646eaf1530fa0f922e3cf60b18da48b1c826be21"} Jul 16 16:14:18.516884 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:18.516837 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" podStartSLOduration=1.5168217099999999 podStartE2EDuration="1.51682171s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-16 16:14:17 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:14:18.515000512 +0000 UTC m=+2346.216552402" watchObservedRunningTime="2026-07-16 16:14:18.51682171 +0000 UTC m=+2346.218373601" Jul 16 16:14:21.840796 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:21.840761 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:14:27.391605 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:27.391566 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:27.392132 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:27.391654 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:27.404252 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:27.404229 2553 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:27.557741 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:27.557710 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:14:36.829525 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:36.829487 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:14:57.924086 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:14:57.924053 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:15:02.761419 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:02.761390 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:15:12.972452 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:12.972423 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:15:12.978675 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:12.978652 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:15:12.980608 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:12.980587 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:15:12.986494 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:12.986476 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-jzljd_6ffde159-037f-4b6e-ab53-58f0e26ecae4/ovn-acl-logging/0.log" Jul 16 16:15:16.117067 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:16.117022 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:15:17.781940 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:17.781875 2553 kuberuntime_container.go:873] "Killing container with a grace period" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" podUID="82e21c79-2ad8-498c-8af1-2efc049b12d6" containerName="main" containerID="cri-o://6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3" gracePeriod=45 Jul 16 16:15:18.039364 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.039309 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:15:18.157027 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157000 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157221 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157047 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157221 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157092 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157221 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157122 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cv6xv\" (UniqueName: \"kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157221 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157186 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157434 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157228 2553 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir\") pod \"82e21c79-2ad8-498c-8af1-2efc049b12d6\" (UID: \"82e21c79-2ad8-498c-8af1-2efc049b12d6\") " Jul 16 16:15:18.157485 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157403 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache" (OuterVolumeSpecName: "model-cache") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "model-cache". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:15:18.157526 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157476 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home" (OuterVolumeSpecName: "home") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "home". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:15:18.157564 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.157540 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir" (OuterVolumeSpecName: "tmp-dir") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "tmp-dir". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:15:18.159259 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.159238 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv" (OuterVolumeSpecName: "kube-api-access-cv6xv") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "kube-api-access-cv6xv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 16 16:15:18.159699 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.159676 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm" (OuterVolumeSpecName: "dshm") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "dshm". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 16 16:15:18.159750 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.159682 2553 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs" (OuterVolumeSpecName: "tls-certs") pod "82e21c79-2ad8-498c-8af1-2efc049b12d6" (UID: "82e21c79-2ad8-498c-8af1-2efc049b12d6"). InnerVolumeSpecName "tls-certs". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 16 16:15:18.258883 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258847 2553 reconciler_common.go:299] "Volume detached for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-tmp-dir\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.258883 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258876 2553 reconciler_common.go:299] "Volume detached for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-dshm\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.258883 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258885 2553 reconciler_common.go:299] "Volume detached for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/82e21c79-2ad8-498c-8af1-2efc049b12d6-tls-certs\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.258883 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258892 2553 reconciler_common.go:299] "Volume detached for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-model-cache\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.259143 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258901 2553 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cv6xv\" (UniqueName: \"kubernetes.io/projected/82e21c79-2ad8-498c-8af1-2efc049b12d6-kube-api-access-cv6xv\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.259143 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.258911 2553 reconciler_common.go:299] "Volume detached for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/82e21c79-2ad8-498c-8af1-2efc049b12d6-home\") on node \"ip-10-0-134-168.ec2.internal\" DevicePath \"\"" Jul 16 16:15:18.812295 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.812261 2553 generic.go:356] "Generic (PLEG): container finished" podID="82e21c79-2ad8-498c-8af1-2efc049b12d6" containerID="6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3" exitCode=0 Jul 16 16:15:18.812738 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.812306 2553 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" Jul 16 16:15:18.812738 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.812349 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" event={"ID":"82e21c79-2ad8-498c-8af1-2efc049b12d6","Type":"ContainerDied","Data":"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3"} Jul 16 16:15:18.812738 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.812394 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77" event={"ID":"82e21c79-2ad8-498c-8af1-2efc049b12d6","Type":"ContainerDied","Data":"f8d379b9ae1befef07a9b413646eaf1530fa0f922e3cf60b18da48b1c826be21"} Jul 16 16:15:18.812738 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.812426 2553 scope.go:117] "RemoveContainer" containerID="6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3" Jul 16 16:15:18.821884 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.821865 2553 scope.go:117] "RemoveContainer" containerID="6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3" Jul 16 16:15:18.822120 ip-10-0-134-168 kubenswrapper[2553]: E0716 16:15:18.822101 2553 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3\": container with ID starting with 6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3 not found: ID does not exist" containerID="6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3" Jul 16 16:15:18.822165 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.822128 2553 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3"} err="failed to get container status \"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3\": rpc error: code = NotFound desc = could not find container \"6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3\": container with ID starting with 6d428066f35485ea913407d4632ce8c8d6dd7f4b828adfb01684356b6487afc3 not found: ID does not exist" Jul 16 16:15:18.836740 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.836711 2553 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:15:18.841016 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.840996 2553 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["ai-tenant-e2e-shared-b-590f9a/rate-test-model-590f9a-kserve-78cd9f955f-6sq77"] Jul 16 16:15:18.888308 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:18.888280 2553 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="82e21c79-2ad8-498c-8af1-2efc049b12d6" path="/var/lib/kubelet/pods/82e21c79-2ad8-498c-8af1-2efc049b12d6/volumes" Jul 16 16:15:28.445850 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:28.445817 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:15:41.016376 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:41.016295 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-xlpt9"] Jul 16 16:15:44.557593 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:44.557563 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz_59077717-e468-4965-8e85-b8d76a7f99a4/manager/0.log" Jul 16 16:15:44.568184 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:44.568163 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz_59077717-e468-4965-8e85-b8d76a7f99a4/copy-manifests/0.log" Jul 16 16:15:46.435854 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.435826 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/util/0.log" Jul 16 16:15:46.443461 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.443419 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/pull/0.log" Jul 16 16:15:46.450822 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.450801 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/extract/0.log" Jul 16 16:15:46.578248 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.578224 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/util/0.log" Jul 16 16:15:46.585559 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.585528 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/pull/0.log" Jul 16 16:15:46.592791 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.592773 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/extract/0.log" Jul 16 16:15:46.709994 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.709934 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/util/0.log" Jul 16 16:15:46.718300 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.718284 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/pull/0.log" Jul 16 16:15:46.725975 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.725959 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/extract/0.log" Jul 16 16:15:46.845274 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.845253 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/pull/0.log" Jul 16 16:15:46.852098 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.852077 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/extract/0.log" Jul 16 16:15:46.858915 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:46.858898 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/util/0.log" Jul 16 16:15:47.799520 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:47.799491 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xlpt9_165e6e95-37c3-42d2-ad2d-50b2fbbfe388/limitador/0.log" Jul 16 16:15:48.422374 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:48.422342 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8pmrp_de9b7f04-7adc-4269-9564-b019319c30a1/discovery/0.log" Jul 16 16:15:49.173194 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:49.173165 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75bd6b5b78-gc5mg_59387a74-b93a-49b7-86a6-4a6223fb9191/router/0.log" Jul 16 16:15:49.665315 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:49.665282 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-bs5nf_02a519b2-2540-4798-aefe-ba0618933bc4/main/0.log" Jul 16 16:15:49.896926 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:49.896895 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-6b86dbl8hkl_a47b8a1b-ba6b-4073-b7f1-25dfb8dab3cc/main/0.log" Jul 16 16:15:50.019533 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:50.019460 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-5jrc5_79ff5919-8b51-46d6-98e8-d6c410893337/main/0.log" Jul 16 16:15:57.117949 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:57.117920 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gtpqd_116501b0-acde-4960-b464-465ccd1d27a6/global-pull-secret-syncer/0.log" Jul 16 16:15:57.304584 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:57.304556 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-gbvcg_e3e5d41d-c3d4-423c-88a5-c15a648843dc/konnectivity-agent/0.log" Jul 16 16:15:57.391093 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:15:57.391022 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-168.ec2.internal_f8e6baf277b94b5597c0f3301bb4ca1d/haproxy/0.log" Jul 16 16:16:01.738885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.738856 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/extract/0.log" Jul 16 16:16:01.769337 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.769304 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/util/0.log" Jul 16 16:16:01.800342 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.800319 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759mflm4_e8246888-9854-464c-bd31-c23354bd53fc/pull/0.log" Jul 16 16:16:01.841545 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.841486 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/extract/0.log" Jul 16 16:16:01.863862 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.863833 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/util/0.log" Jul 16 16:16:01.891688 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.891664 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0glcrj_de7db5a1-ece4-4ec1-bb0f-b72864d45a18/pull/0.log" Jul 16 16:16:01.924085 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.924059 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/extract/0.log" Jul 16 16:16:01.950968 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.950942 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/util/0.log" Jul 16 16:16:01.981395 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:01.981367 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73qdwjh_29526d22-48bd-4a0a-b8fe-326837f0827f/pull/0.log" Jul 16 16:16:02.006386 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:02.006322 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/extract/0.log" Jul 16 16:16:02.037000 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:02.036976 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/util/0.log" Jul 16 16:16:02.063796 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:02.063771 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef18m7b5_e68debf5-4acc-4f14-92e6-b07a83cfd478/pull/0.log" Jul 16 16:16:02.840976 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:02.840948 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-xlpt9_165e6e95-37c3-42d2-ad2d-50b2fbbfe388/limitador/0.log" Jul 16 16:16:05.082615 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:05.082583 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-7v7l6_a99c00fc-e6ba-450a-9505-c614036a222c/cluster-monitoring-operator/0.log" Jul 16 16:16:05.444762 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:05.444732 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrf7q_46daf924-da20-4105-9802-439becec5cfb/node-exporter/0.log" Jul 16 16:16:05.467454 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:05.467427 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrf7q_46daf924-da20-4105-9802-439becec5cfb/kube-rbac-proxy/0.log" Jul 16 16:16:05.491705 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:05.491686 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-hrf7q_46daf924-da20-4105-9802-439becec5cfb/init-textfile/0.log" Jul 16 16:16:05.872137 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:05.872108 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-r2qrr_34899729-f8b5-4a85-a55c-c5762c5fd993/prometheus-operator-admission-webhook/0.log" Jul 16 16:16:06.700724 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.700695 2553 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw"] Jul 16 16:16:06.701086 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.701033 2553 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="82e21c79-2ad8-498c-8af1-2efc049b12d6" containerName="main" Jul 16 16:16:06.701086 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.701045 2553 state_mem.go:107] "Deleted CPUSet assignment" podUID="82e21c79-2ad8-498c-8af1-2efc049b12d6" containerName="main" Jul 16 16:16:06.701162 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.701096 2553 memory_manager.go:368] "RemoveStaleState removing state" podUID="82e21c79-2ad8-498c-8af1-2efc049b12d6" containerName="main" Jul 16 16:16:06.704224 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.704206 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.707228 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.707197 2553 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-dnqzv\"/\"default-dockercfg-5jg58\"" Jul 16 16:16:06.708481 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.708427 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnqzv\"/\"openshift-service-ca.crt\"" Jul 16 16:16:06.708895 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.708877 2553 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-dnqzv\"/\"kube-root-ca.crt\"" Jul 16 16:16:06.712397 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.711717 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw"] Jul 16 16:16:06.870943 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.870916 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-proc\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.871104 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.870950 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-lib-modules\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.871104 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.870972 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fpr74\" (UniqueName: \"kubernetes.io/projected/31f66ce3-66fd-48ba-a2cf-754df6015a11-kube-api-access-fpr74\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.871104 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.871061 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-sys\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.871104 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.871104 2553 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-podres\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972071 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.971982 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-lib-modules\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972071 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972028 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fpr74\" (UniqueName: \"kubernetes.io/projected/31f66ce3-66fd-48ba-a2cf-754df6015a11-kube-api-access-fpr74\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972071 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972063 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-sys\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972089 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-podres\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972142 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-lib-modules\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972157 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-sys\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972256 2553 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-proc\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972284 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-podres\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.972347 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.972337 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/31f66ce3-66fd-48ba-a2cf-754df6015a11-proc\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:06.981829 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:06.981800 2553 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fpr74\" (UniqueName: \"kubernetes.io/projected/31f66ce3-66fd-48ba-a2cf-754df6015a11-kube-api-access-fpr74\") pod \"perf-node-gather-daemonset-gcbfw\" (UID: \"31f66ce3-66fd-48ba-a2cf-754df6015a11\") " pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:07.015862 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:07.015840 2553 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:07.154763 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:07.154741 2553 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw"] Jul 16 16:16:07.156605 ip-10-0-134-168 kubenswrapper[2553]: W0716 16:16:07.156580 2553 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod31f66ce3_66fd_48ba_a2cf_754df6015a11.slice/crio-b641d731a4fdec276e5c4d62915475dbc8f082f3e8885efccb39d95dc8a8e748 WatchSource:0}: Error finding container b641d731a4fdec276e5c4d62915475dbc8f082f3e8885efccb39d95dc8a8e748: Status 404 returned error can't find the container with id b641d731a4fdec276e5c4d62915475dbc8f082f3e8885efccb39d95dc8a8e748 Jul 16 16:16:07.953204 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:07.953177 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/1.log" Jul 16 16:16:07.958603 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:07.958583 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7866497494-d4qkh_c5a56410-8aa7-414a-b2ac-3614a129bf35/console-operator/2.log" Jul 16 16:16:08.049861 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:08.049832 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" event={"ID":"31f66ce3-66fd-48ba-a2cf-754df6015a11","Type":"ContainerStarted","Data":"363310a74e07e7ba9317642a396d615ac4e44aef52315f7cd08f44bfcd25ed0d"} Jul 16 16:16:08.049995 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:08.049879 2553 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" event={"ID":"31f66ce3-66fd-48ba-a2cf-754df6015a11","Type":"ContainerStarted","Data":"b641d731a4fdec276e5c4d62915475dbc8f082f3e8885efccb39d95dc8a8e748"} Jul 16 16:16:08.049995 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:08.049992 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:08.074910 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:08.074867 2553 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" podStartSLOduration=2.074853908 podStartE2EDuration="2.074853908s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-16 16:16:06 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-16 16:16:08.073828087 +0000 UTC m=+2455.775379984" watchObservedRunningTime="2026-07-16 16:16:08.074853908 +0000 UTC m=+2455.776405798" Jul 16 16:16:08.477952 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:08.477926 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-68c9cbb497-vsj6z_3cc3958a-877f-4c4e-95f7-4eb756361fe2/download-server/0.log" Jul 16 16:16:09.106996 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:09.106965 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-mh6lp_80c6cade-892f-4f61-9712-53a749f66d26/volume-data-source-validator/0.log" Jul 16 16:16:10.137086 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:10.137059 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4qgdz_29e3315a-0858-414b-9521-47caafd81e3a/dns/0.log" Jul 16 16:16:10.170852 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:10.170825 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-4qgdz_29e3315a-0858-414b-9521-47caafd81e3a/kube-rbac-proxy/0.log" Jul 16 16:16:10.491914 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:10.491842 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-z96xk_84ea9286-2f1d-4d42-848e-e48204a86073/dns-node-resolver/0.log" Jul 16 16:16:11.106501 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:11.106471 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-4zz9k_8338837b-8db4-4083-bab5-0a846db22992/node-ca/0.log" Jul 16 16:16:12.291862 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:12.291833 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8pmrp_de9b7f04-7adc-4269-9564-b019319c30a1/discovery/0.log" Jul 16 16:16:12.616759 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:12.616733 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75bd6b5b78-gc5mg_59387a74-b93a-49b7-86a6-4a6223fb9191/router/0.log" Jul 16 16:16:13.388202 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:13.388175 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-vl47k_3af65aa6-9f3c-4000-a63e-a03a4134d0cd/serve-healthcheck-canary/0.log" Jul 16 16:16:13.901631 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:13.901594 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-fzp89_018f2da5-c589-4c28-988f-432c9643da7c/insights-operator/0.log" Jul 16 16:16:13.902273 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:13.902254 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-68d9dd94cf-fzp89_018f2da5-c589-4c28-988f-432c9643da7c/insights-operator/1.log" Jul 16 16:16:14.019602 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:14.019569 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpmjg_2b974907-cb45-4089-be70-974082496bba/kube-rbac-proxy/0.log" Jul 16 16:16:14.061734 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:14.061712 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpmjg_2b974907-cb45-4089-be70-974082496bba/exporter/0.log" Jul 16 16:16:14.064707 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:14.064685 2553 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-dnqzv/perf-node-gather-daemonset-gcbfw" Jul 16 16:16:14.116825 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:14.116802 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-fpmjg_2b974907-cb45-4089-be70-974082496bba/extractor/0.log" Jul 16 16:16:17.602885 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:17.602860 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz_59077717-e468-4965-8e85-b8d76a7f99a4/manager/0.log" Jul 16 16:16:17.648267 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:17.648248 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-7b5b94d4fb-jbmzz_59077717-e468-4965-8e85-b8d76a7f99a4/copy-manifests/0.log" Jul 16 16:16:17.929379 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:17.929349 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8pmrp_de9b7f04-7adc-4269-9564-b019319c30a1/discovery/0.log" Jul 16 16:16:18.173690 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:18.173663 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75bd6b5b78-gc5mg_59387a74-b93a-49b7-86a6-4a6223fb9191/router/0.log" Jul 16 16:16:19.039560 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:19.039536 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-75498485d8-xvjkn_2653fa9e-c76d-40aa-8469-de6e2e2e6f22/manager/0.log" Jul 16 16:16:19.456837 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:19.456807 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_istiod-openshift-gateway-94bb8fbfd-8pmrp_de9b7f04-7adc-4269-9564-b019319c30a1/discovery/0.log" Jul 16 16:16:19.723007 ip-10-0-134-168 kubenswrapper[2553]: I0716 16:16:19.722940 2553 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-75bd6b5b78-gc5mg_59387a74-b93a-49b7-86a6-4a6223fb9191/router/0.log"