Jul 15 16:37:08.754167 ip-10-0-138-140 systemd[1]: Starting Kubernetes Kubelet... Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.131684 2566 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137342 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137357 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137361 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137364 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137368 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137371 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137374 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:09.140062 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137378 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137381 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137384 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137386 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137389 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137397 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137400 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137404 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137408 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137411 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137414 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137417 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137419 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137422 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137425 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137428 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137430 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137433 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137437 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:09.141309 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137442 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137445 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137448 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137450 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137454 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137456 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137459 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137461 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137464 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137476 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137481 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137484 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137487 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137490 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137493 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137496 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137500 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137502 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137505 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137507 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:09.141927 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137510 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137513 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137515 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137518 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137520 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137523 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137525 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137528 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137530 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137532 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137535 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137539 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137542 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137545 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137547 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137550 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137553 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137556 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137559 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137561 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:09.142450 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137564 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137566 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137569 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137571 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137574 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137576 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137579 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137581 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137584 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137586 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137589 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137592 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137595 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137597 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137600 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137602 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137605 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137607 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137610 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137612 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:09.143000 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137615 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137617 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137620 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137623 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137626 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137628 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137631 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137633 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137636 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137638 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137641 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137643 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.137646 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139606 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139614 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139619 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139624 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139627 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139630 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:09.143512 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139633 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139636 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139639 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139643 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139646 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139649 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139652 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139654 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139657 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139661 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139663 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139666 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139669 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139671 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139674 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139677 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139680 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139682 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139685 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139688 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:09.144079 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139690 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139693 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139696 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139698 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139700 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139703 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139706 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139709 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139711 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139714 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139717 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139719 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139722 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139724 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139727 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139730 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139732 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139735 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139737 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:09.144623 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139740 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139743 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139746 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139748 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139750 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139753 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139755 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139758 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139760 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139763 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139765 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139768 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139770 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139773 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139775 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139778 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139780 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139783 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139785 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139789 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:09.145181 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139791 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139794 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139796 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139799 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139801 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139803 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139806 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139809 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139813 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139815 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139818 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139822 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139825 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139828 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139831 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139834 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139837 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139839 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139843 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:09.145779 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139847 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139849 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139852 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139855 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139857 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139860 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139862 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139865 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139868 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139870 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139873 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139875 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139878 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139880 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.139883 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140286 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140291 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140294 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140297 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140300 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:09.146332 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140302 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140305 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140307 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140311 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140314 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140316 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140320 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140322 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140325 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140327 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140330 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140333 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140335 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140338 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140340 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140343 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140345 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140348 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140350 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140353 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:09.146887 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140356 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140358 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140361 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140363 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140366 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140369 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140371 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140374 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140377 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140380 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140382 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140385 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140390 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140392 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140395 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140397 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140400 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140403 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140405 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140408 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:09.147477 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140410 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140413 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140416 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140418 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140421 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140423 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140426 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140428 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140431 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140434 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140436 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140440 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140443 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140445 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140447 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140450 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140452 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140455 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140459 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:09.148053 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140463 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140466 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140469 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140471 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140474 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140479 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140483 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140485 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140488 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140491 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140493 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140496 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140498 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140501 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140503 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140506 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140508 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140511 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140514 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:09.148830 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140518 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140521 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140524 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140527 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140529 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140532 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140534 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140537 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140540 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140542 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140545 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140547 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140550 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140552 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140555 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.140557 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140628 2566 flags.go:64] FLAG: --address="0.0.0.0" Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140635 2566 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140645 2566 flags.go:64] FLAG: --anonymous-auth="true" Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140649 2566 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140654 2566 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 15 16:37:09.149659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140657 2566 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140661 2566 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140665 2566 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140669 2566 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140672 2566 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140676 2566 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140679 2566 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140682 2566 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140685 2566 flags.go:64] FLAG: --cgroup-root="" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140688 2566 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140691 2566 flags.go:64] FLAG: --client-ca-file="" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140694 2566 flags.go:64] FLAG: --cloud-provider="external" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140697 2566 flags.go:64] FLAG: --cluster-dns="[]" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140701 2566 flags.go:64] FLAG: --cluster-domain="" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140703 2566 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140706 2566 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140710 2566 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140713 2566 flags.go:64] FLAG: --container-log-max-files="5" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140717 2566 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140720 2566 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140723 2566 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140726 2566 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140729 2566 flags.go:64] FLAG: --contention-profiling="false" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140732 2566 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 15 16:37:09.150298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140735 2566 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140738 2566 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140741 2566 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140745 2566 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140748 2566 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140752 2566 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140755 2566 flags.go:64] FLAG: --enable-load-reader="false" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140758 2566 flags.go:64] FLAG: --enable-server="true" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140761 2566 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140764 2566 flags.go:64] FLAG: --event-burst="100" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140767 2566 flags.go:64] FLAG: --event-qps="50" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140771 2566 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140774 2566 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140777 2566 flags.go:64] FLAG: --eviction-hard="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140781 2566 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140784 2566 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140787 2566 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140790 2566 flags.go:64] FLAG: --eviction-soft="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140793 2566 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140796 2566 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140798 2566 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140802 2566 flags.go:64] FLAG: --experimental-mounter-path="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140805 2566 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140808 2566 flags.go:64] FLAG: --fail-swap-on="true" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140811 2566 flags.go:64] FLAG: --feature-gates="" Jul 15 16:37:09.150897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140815 2566 flags.go:64] FLAG: --file-check-frequency="20s" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140817 2566 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140820 2566 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140824 2566 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140827 2566 flags.go:64] FLAG: --healthz-port="10248" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140830 2566 flags.go:64] FLAG: --help="false" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140833 2566 flags.go:64] FLAG: --hostname-override="ip-10-0-138-140.ec2.internal" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140835 2566 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140839 2566 flags.go:64] FLAG: --http-check-frequency="20s" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140842 2566 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140845 2566 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140849 2566 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140853 2566 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140856 2566 flags.go:64] FLAG: --image-service-endpoint="" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140859 2566 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140861 2566 flags.go:64] FLAG: --kube-api-burst="100" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140864 2566 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140867 2566 flags.go:64] FLAG: --kube-api-qps="50" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140870 2566 flags.go:64] FLAG: --kube-reserved="" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140873 2566 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140877 2566 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140880 2566 flags.go:64] FLAG: --kubelet-cgroups="" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140882 2566 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140887 2566 flags.go:64] FLAG: --lock-file="" Jul 15 16:37:09.151647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140891 2566 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140893 2566 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140896 2566 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140916 2566 flags.go:64] FLAG: --log-json-split-stream="false" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140920 2566 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140922 2566 flags.go:64] FLAG: --log-text-split-stream="false" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140925 2566 flags.go:64] FLAG: --logging-format="text" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140928 2566 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140931 2566 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140934 2566 flags.go:64] FLAG: --manifest-url="" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140937 2566 flags.go:64] FLAG: --manifest-url-header="" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140941 2566 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140945 2566 flags.go:64] FLAG: --max-open-files="1000000" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140949 2566 flags.go:64] FLAG: --max-pods="110" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140952 2566 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140955 2566 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140959 2566 flags.go:64] FLAG: --memory-manager-policy="None" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140962 2566 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140965 2566 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140968 2566 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140972 2566 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140979 2566 flags.go:64] FLAG: --node-status-max-images="50" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140982 2566 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140985 2566 flags.go:64] FLAG: --oom-score-adj="-999" Jul 15 16:37:09.152536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140988 2566 flags.go:64] FLAG: --pod-cidr="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140991 2566 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:b04c0b1f73af4d021dd8f0b51bfeb302c24482c093de5f2c43654ab56c506926" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.140997 2566 flags.go:64] FLAG: --pod-manifest-path="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141000 2566 flags.go:64] FLAG: --pod-max-pids="-1" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141003 2566 flags.go:64] FLAG: --pods-per-core="0" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141006 2566 flags.go:64] FLAG: --port="10250" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141009 2566 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141014 2566 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-07f99ba8f5f479e4e" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141017 2566 flags.go:64] FLAG: --qos-reserved="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141020 2566 flags.go:64] FLAG: --read-only-port="10255" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141023 2566 flags.go:64] FLAG: --register-node="true" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141026 2566 flags.go:64] FLAG: --register-with-taints="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141029 2566 flags.go:64] FLAG: --registry-burst="10" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141032 2566 flags.go:64] FLAG: --registry-qps="5" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141035 2566 flags.go:64] FLAG: --reserved-cpus="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141038 2566 flags.go:64] FLAG: --reserved-memory="" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141041 2566 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141044 2566 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141047 2566 flags.go:64] FLAG: --rotate-certificates="false" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141050 2566 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141053 2566 flags.go:64] FLAG: --runonce="false" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141055 2566 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141059 2566 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141062 2566 flags.go:64] FLAG: --seccomp-default="false" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141064 2566 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 15 16:37:09.153221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141068 2566 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141071 2566 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141074 2566 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141077 2566 flags.go:64] FLAG: --storage-driver-password="root" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141082 2566 flags.go:64] FLAG: --storage-driver-secure="false" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141085 2566 flags.go:64] FLAG: --storage-driver-table="stats" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141087 2566 flags.go:64] FLAG: --storage-driver-user="root" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141090 2566 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141093 2566 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141096 2566 flags.go:64] FLAG: --system-cgroups="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141099 2566 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141104 2566 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141107 2566 flags.go:64] FLAG: --tls-cert-file="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141110 2566 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141115 2566 flags.go:64] FLAG: --tls-min-version="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141118 2566 flags.go:64] FLAG: --tls-private-key-file="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141121 2566 flags.go:64] FLAG: --topology-manager-policy="none" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141124 2566 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141127 2566 flags.go:64] FLAG: --topology-manager-scope="container" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141130 2566 flags.go:64] FLAG: --v="2" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141134 2566 flags.go:64] FLAG: --version="false" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141138 2566 flags.go:64] FLAG: --vmodule="" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141142 2566 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.141145 2566 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 15 16:37:09.154605 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141276 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141279 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141282 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141286 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141289 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141292 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141294 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141297 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141299 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141302 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141304 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141307 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141311 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141313 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141316 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141319 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141321 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141324 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141327 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141329 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:09.155483 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141332 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141334 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141338 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141341 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141344 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141346 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141349 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141352 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141354 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141356 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141359 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141361 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141364 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141367 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141371 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141373 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141376 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141380 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141383 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:09.288491 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141386 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:09.208190 ip-10-0-138-140 systemd[1]: Started Kubernetes Kubelet. Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141389 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141391 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141394 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141397 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141401 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141403 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141406 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141409 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141411 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141414 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141416 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141419 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141422 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141424 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141430 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141434 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141437 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141440 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:09.289171 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141443 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141446 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141449 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141451 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141454 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141456 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141459 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141461 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141464 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141466 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141476 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141479 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141482 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141484 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141487 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141490 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141492 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141495 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141498 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141501 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:09.290341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141504 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141506 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141509 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141511 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141514 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141516 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141519 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141522 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141525 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141528 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141531 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141534 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141537 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141539 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141542 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141544 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141547 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141549 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141551 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141554 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:09.290996 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.141556 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.142368 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.150567 2566 server.go:529] "Kubelet version" kubeletVersion="v1.34.9" Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.150583 2566 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.151355 2566 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.151367 2566 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151423 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151429 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151432 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151435 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151438 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151442 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151445 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151448 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:09.291664 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151450 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151453 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151456 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151459 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151462 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151464 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151467 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151469 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151473 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151476 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151478 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151482 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151485 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151487 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151490 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151493 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151497 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151500 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151503 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:09.292135 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151506 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151508 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151511 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151513 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151516 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151519 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151521 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151524 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151526 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151529 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151532 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151534 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151537 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151539 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151542 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151545 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151547 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151550 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151553 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151556 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:09.292767 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151559 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151562 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151564 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151567 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151569 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151572 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151574 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151577 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151580 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151582 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151585 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151587 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151590 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151593 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151595 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151598 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151601 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151603 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151606 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151609 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:09.293819 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151611 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151613 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151616 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151619 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151621 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151624 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151627 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151629 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151632 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151635 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151638 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151641 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151645 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151648 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151651 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151654 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151657 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151660 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151663 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:10.596262 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151666 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151668 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151671 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151674 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151677 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151679 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151682 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151685 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151687 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151690 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151693 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151696 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151700 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.151706 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151821 2566 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 15 16:37:10.597249 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151826 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151829 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151832 2566 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151835 2566 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151838 2566 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151840 2566 feature_gate.go:328] unrecognized feature gate: Example2 Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151843 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151845 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151848 2566 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151850 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151853 2566 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151856 2566 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151858 2566 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151861 2566 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151863 2566 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151866 2566 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151869 2566 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151872 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151875 2566 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151878 2566 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 15 16:37:10.597911 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151880 2566 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151883 2566 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151886 2566 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151889 2566 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151891 2566 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151894 2566 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151896 2566 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151899 2566 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151916 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151919 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151922 2566 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151925 2566 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151927 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151930 2566 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151932 2566 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151935 2566 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151938 2566 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151940 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151943 2566 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 15 16:37:10.598586 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151945 2566 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151948 2566 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151950 2566 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151953 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151955 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151958 2566 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151961 2566 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151964 2566 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151967 2566 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151970 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151974 2566 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151977 2566 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151981 2566 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151984 2566 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151987 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151991 2566 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151994 2566 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.151997 2566 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152000 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 15 16:37:10.599152 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152002 2566 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152005 2566 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152008 2566 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152011 2566 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152014 2566 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152016 2566 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152019 2566 feature_gate.go:328] unrecognized feature gate: Example Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152021 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152024 2566 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152027 2566 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152029 2566 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152031 2566 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152034 2566 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152036 2566 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152039 2566 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152042 2566 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152044 2566 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152047 2566 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152049 2566 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152052 2566 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 15 16:37:10.599836 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152055 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152058 2566 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152060 2566 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152063 2566 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152066 2566 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152068 2566 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152071 2566 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152074 2566 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152076 2566 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152079 2566 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152081 2566 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152084 2566 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152087 2566 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152089 2566 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152092 2566 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152094 2566 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152099 2566 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152102 2566 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152105 2566 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 15 16:37:10.600402 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:09.152108 2566 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.152113 2566 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.152237 2566 server.go:962] "Client rotation is on, will bootstrap in background" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.154699 2566 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.156126 2566 server.go:1019] "Starting client certificate rotation" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.159075 2566 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.159894 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.159944 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.163300 2566 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.177311 2566 log.go:25] "Validated CRI v1 runtime API" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.179369 2566 log.go:25] "Validated CRI v1 image API" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.182440 2566 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 15 16:37:10.601057 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.184829 2566 fs.go:135] Filesystem UUIDs: map[16c9766e-4f56-43eb-8f56-e8f43fa20d1d:/dev/nvme0n1p4 6bf084db-01fe-414f-9be4-5ba2e949a781:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2] Jul 15 16:37:10.601450 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.184844 2566 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.191073 2566 manager.go:216] Machine: {Timestamp:2026-07-15 16:37:09.190508539 +0000 UTC m=+0.322300935 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3103580 MemoryCapacity:33164488704 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec221734ee7cf8e58e8e7554c5b5d059 SystemUUID:ec221734-ee7c-f8e5-8e8e-7554c5b5d059 BootID:45d711b1-6567-4e4f-853a-a8dd0f4bd8fe Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582246400 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582242304 Type:vfs Inodes:4048399 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632898560 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:82:39:fc:c1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:82:39:fc:c1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:82:64:d5:ea:66:06 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164488704 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.191223 2566 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.191345 2566 manager.go:232] Version: {KernelVersion:5.14.0-570.126.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260702-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.194227 2566 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.194395 2566 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-140.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.195049 2566 topology_manager.go:138] "Creating topology manager with none policy" Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.195060 2566 container_manager_linux.go:309] "Creating device plugin manager" Jul 15 16:37:10.601626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.195073 2566 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.195086 2566 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.195169 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.196181 2566 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.198875 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.198990 2566 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199017 2566 kubelet.go:486] "Attempting to sync node with API server" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199027 2566 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199037 2566 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199046 2566 kubelet.go:398] "Adding apiserver pod source" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199054 2566 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199384 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.199400 2566 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.201261 2566 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.10-2.rhaos4.21.git83ee25b.el9" apiVersion="v1" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203073 2566 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203101 2566 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203745 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203758 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203764 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203770 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203778 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203783 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203792 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203797 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203804 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203811 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203820 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.203825 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.204574 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.204581 2566 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.207579 2566 server.go:1268] "Started kubelet" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.210930 2566 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.210989 2566 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.211008 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.211016 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.211026 2566 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.211052 2566 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-140.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.212466 2566 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 15 16:37:10.601928 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.212740 2566 server.go:310] "Adding debug handlers to kubelet server" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.212927 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-140.ec2.internal.18c283e6afd0e39b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-140.ec2.internal,UID:ip-10-0-138-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-140.ec2.internal,},FirstTimestamp:2026-07-15 16:37:09.207557019 +0000 UTC m=+0.339349415,LastTimestamp:2026-07-15 16:37:09.207557019 +0000 UTC m=+0.339349415,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-140.ec2.internal,}" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.214501 2566 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.215272 2566 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.216617 2566 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.217475 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.218005 2566 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.218005 2566 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.218028 2566 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.218088 2566 reconstruct.go:97] "Volume reconstruction finished" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.218099 2566 reconciler.go:29] "Reconciler: start to sync state" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.219563 2566 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.219738 2566 factory.go:55] Registering systemd factory Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.219753 2566 factory.go:223] Registration of the systemd container factory successfully Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.219961 2566 factory.go:153] Registering CRI-O factory Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.219972 2566 factory.go:223] Registration of the crio container factory successfully Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.220027 2566 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.220051 2566 factory.go:103] Registering Raw factory Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.220064 2566 manager.go:1196] Started watching for new ooms in manager Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.220436 2566 manager.go:349] Starting recovery of all containers Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.220673 2566 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-140.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.221123 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lvrts" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.230070 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-lvrts" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.231416 2566 manager.go:354] Recovery completed Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.235672 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.237480 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.237501 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.237510 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238061 2566 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238073 2566 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238077 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238088 2566 state_mem.go:36] "Initialized new in-memory state store" Jul 15 16:37:10.602918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238736 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238757 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.238767 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.239003 2566 policy_none.go:49] "None policy: Start" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.239018 2566 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.239031 2566 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.239652 2566 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-140.ec2.internal.18c283e6b199a5bf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-140.ec2.internal,UID:ip-10-0-138-140.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-140.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-140.ec2.internal,},FirstTimestamp:2026-07-15 16:37:09.237491135 +0000 UTC m=+0.369283531,LastTimestamp:2026-07-15 16:37:09.237491135 +0000 UTC m=+0.369283531,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-140.ec2.internal,}" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.239973 2566 policy_none.go:47] "Start" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.273943 2566 manager.go:337] "Starting Device Plugin manager" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.274076 2566 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274089 2566 server.go:87] "Starting device plugin registration server" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274273 2566 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274282 2566 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274373 2566 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274445 2566 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:09.274452 2566 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.274935 2566 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:09.274974 2566 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.490710 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.492598 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.492624 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.492634 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.492657 2566 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.499445 2566 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.499465 2566 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-140.ec2.internal\": node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.515627 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.516846 2566 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.516888 2566 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.516918 2566 kubelet.go:2448] "Starting kubelet main sync loop" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.516952 2566 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.518760 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.603780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.519469 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.617045 2566 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal"] Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.617106 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.618724 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.618750 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.618759 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.618853 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.620018 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.620143 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.620167 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621075 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621101 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621110 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621078 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621178 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.621189 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.622313 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.622356 2566 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.623014 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientMemory" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.623034 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasNoDiskPressure" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.623047 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeHasSufficientPID" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.648875 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-140.ec2.internal\" not found" node="ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.653190 2566 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-140.ec2.internal\" not found" node="ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.719309 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.734905 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.734925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.818742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.734961 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8eeed7395da0aa41ef27a523d3170f4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-140.ec2.internal\" (UID: \"a8eeed7395da0aa41ef27a523d3170f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:10.820664 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.820121 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8eeed7395da0aa41ef27a523d3170f4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-140.ec2.internal\" (UID: \"a8eeed7395da0aa41ef27a523d3170f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836720 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836772 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836776 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a8eeed7395da0aa41ef27a523d3170f4-config\") pod \"kube-apiserver-proxy-ip-10-0-138-140.ec2.internal\" (UID: \"a8eeed7395da0aa41ef27a523d3170f4\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.836794 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/e5033fd10d4e085ca52b1cf7265e8f08-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal\" (UID: \"e5033fd10d4e085ca52b1cf7265e8f08\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:10.920425 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.950879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:10.955786 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.040924 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.021078 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.206827 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.122188 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.222754 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.271271 2566 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.271444 2566 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.323700 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.351157 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-07-14 16:32:10 +0000 UTC" deadline="2027-12-28 07:30:46.05775256 +0000 UTC" Jul 15 16:37:11.393803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.351180 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12734h53m34.706575335s" Jul 15 16:37:11.586678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.424776 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.586678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.525103 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:11.625637 2566 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-140.ec2.internal\" not found" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.656804 2566 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.733096 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.749455 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.751043 2566 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" Jul 15 16:37:11.761447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.760724 2566 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 15 16:37:11.938881 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:11.838047 2566 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:37:12.113382 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.088013 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8eeed7395da0aa41ef27a523d3170f4.slice/crio-9face086bc1c644e09910125252ce3b76c820380790d72bbdc486515e1891c08 WatchSource:0}: Error finding container 9face086bc1c644e09910125252ce3b76c820380790d72bbdc486515e1891c08: Status 404 returned error can't find the container with id 9face086bc1c644e09910125252ce3b76c820380790d72bbdc486515e1891c08 Jul 15 16:37:12.113382 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.097198 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:37:12.288217 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.278340 2566 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:37:12.315752 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.315729 2566 apiserver.go:52] "Watching apiserver" Jul 15 16:37:12.323179 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.323162 2566 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 15 16:37:12.323539 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.323519 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-gv5w9","kube-system/konnectivity-agent-bjvfb","kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal","openshift-dns/node-resolver-w4t9b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal","openshift-multus/multus-nhbll","openshift-multus/network-metrics-daemon-rq8qh","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn","openshift-cluster-node-tuning-operator/tuned-xdnkr","openshift-image-registry/node-ca-g6ldc","openshift-multus/multus-additional-cni-plugins-cm6xf","openshift-network-diagnostics/network-check-target-vph5l","openshift-network-operator/iptables-alerter-qv4bf","openshift-ovn-kubernetes/ovnkube-node-z2n2r"] Jul 15 16:37:12.325999 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.325984 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.326061 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.326045 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:12.326995 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.326982 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.328139 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.328123 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.329687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.329668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.330732 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.330709 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.330990 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.330975 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.331169 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.331156 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 15 16:37:12.331336 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.331311 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 15 16:37:12.331592 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.331579 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 15 16:37:12.331781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.331766 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-666td\"" Jul 15 16:37:12.333391 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333373 2566 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 15 16:37:12.333457 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333445 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.333507 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333470 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.333736 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333669 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 15 16:37:12.333868 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333823 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 15 16:37:12.333932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333827 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 15 16:37:12.334014 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.333997 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.334166 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.334146 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.334276 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.334264 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.334389 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.334372 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-mfw9d\"" Jul 15 16:37:12.334461 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.334435 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 15 16:37:12.335012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.334990 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.335448 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.335426 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.335543 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.335494 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.336144 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.336127 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-m754l\"" Jul 15 16:37:12.336144 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.336137 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 15 16:37:12.337949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.336409 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.337949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.337889 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.337949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.337899 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.338833 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.338967 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339039 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.338975 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-mfmmr\"" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339220 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.339254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339244 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.339568 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339282 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.339568 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339320 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 15 16:37:12.339568 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339426 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-9nd45\"" Jul 15 16:37:12.339791 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339775 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-k8v5m\"" Jul 15 16:37:12.339791 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339785 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-928t5\"" Jul 15 16:37:12.339791 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339774 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 15 16:37:12.339950 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339840 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-npjn5\"" Jul 15 16:37:12.339950 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.339932 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 15 16:37:12.340290 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.340276 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.340395 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.340344 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:12.341492 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.341477 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:12.341572 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.341534 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:12.343125 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.343111 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 15 16:37:12.343204 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.343165 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-5s8fd\"" Jul 15 16:37:12.343266 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.343239 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 15 16:37:12.343337 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.343277 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 15 16:37:12.348604 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348588 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.348694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348610 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-log-socket\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.348694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348630 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-26r4j\" (UniqueName: \"kubernetes.io/projected/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-kube-api-access-26r4j\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.348694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348645 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cafe0e2-688f-4634-9fe6-a039e8afc21b-hosts-file\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.348694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jbfz\" (UniqueName: \"kubernetes.io/projected/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kube-api-access-6jbfz\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.348694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348675 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-system-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348695 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-etc-kubernetes\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348734 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwv56\" (UniqueName: \"kubernetes.io/projected/6b6fa6e5-895c-469e-8854-d5dfffec507c-kube-api-access-dwv56\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348772 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xtj2j\" (UniqueName: \"kubernetes.io/projected/0a75565a-dc09-41c9-92a3-0f3bab87860f-kube-api-access-xtj2j\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348789 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-modprobe-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-systemd\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348819 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-sys\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348834 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-tmp\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348855 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwzqw\" (UniqueName: \"kubernetes.io/projected/075d2005-422a-4daa-82bb-e05e695476c0-kube-api-access-vwzqw\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.348876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348870 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348884 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-netns\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348907 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348943 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-registration-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348972 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-host\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.348994 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-config\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-cni-binary-copy\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349029 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-host-slash\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349053 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-cnibin\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349082 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-netd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v5wj\" (UniqueName: \"kubernetes.io/projected/3cafe0e2-688f-4634-9fe6-a039e8afc21b-kube-api-access-9v5wj\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349124 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-k8s-cni-cncf-io\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349182 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-netns\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349219 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-multus-certs\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349245 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-etc-selinux\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349288 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-conf-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-var-lib-kubelet\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349341 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-system-cni-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349358 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349372 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-env-overrides\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-hostroot\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349409 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysconfig\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349427 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-conf\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-etc-tuned\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349457 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-os-release\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349489 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-socket-dir-parent\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349504 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-bin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.349745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-multus\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349567 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-os-release\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-systemd-units\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349618 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-systemd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349637 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-kubelet\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349701 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovn-node-metrics-cert\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349716 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cafe0e2-688f-4634-9fe6-a039e8afc21b-tmp-dir\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-etc-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349776 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-node-log\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bgw86\" (UniqueName: \"kubernetes.io/projected/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-kube-api-access-bgw86\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-socket-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349852 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-device-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349873 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-sys-fs\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.350160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349895 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-cnibin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349934 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349951 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-slash\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349967 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.349984 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-bin\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350088 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-script-lib\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350115 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-kubelet\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350138 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350159 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-lib-modules\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350180 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-iptables-alerter-script\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350214 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350238 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-kubernetes\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.350625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350259 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.351077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-run\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.351077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350301 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f4tzk\" (UniqueName: \"kubernetes.io/projected/9ff0fb0c-56b8-4d12-b30f-99e568bba580-kube-api-access-f4tzk\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.351077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350340 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-var-lib-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.351077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350362 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-ovn\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.351077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.350382 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-daemon-config\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.351994 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.351974 2566 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 15 16:37:12.383724 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.383708 2566 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-xzpqd" Jul 15 16:37:12.392477 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.392458 2566 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-xzpqd" Jul 15 16:37:12.434500 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.434483 2566 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 15 16:37:12.450998 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.450974 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451005 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-slash\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451022 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451038 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451055 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-bin\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451076 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-script-lib\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-kubelet\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451107 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451117 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451121 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451133 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-bin\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451178 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-kubelet\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451193 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-slash\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451220 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-lib-modules\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451226 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-dbus\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451279 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-iptables-alerter-script\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451297 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-lib-modules\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451304 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451357 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-kubernetes\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.451396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451377 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451393 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc95bc2d-0c04-41dc-b889-b33014478432-host\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451416 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc95bc2d-0c04-41dc-b889-b33014478432-serviceca\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-kubernetes\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451444 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-run\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451482 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f4tzk\" (UniqueName: \"kubernetes.io/projected/9ff0fb0c-56b8-4d12-b30f-99e568bba580-kube-api-access-f4tzk\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451497 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-var-lib-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-ovn\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451535 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-daemon-config\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451558 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451579 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-log-socket\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451609 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-26r4j\" (UniqueName: \"kubernetes.io/projected/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-kube-api-access-26r4j\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451612 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.452035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451630 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cafe0e2-688f-4634-9fe6-a039e8afc21b-hosts-file\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451612 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-tuning-conf-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451652 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6jbfz\" (UniqueName: \"kubernetes.io/projected/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kube-api-access-6jbfz\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451675 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-system-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451696 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-etc-kubernetes\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451719 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dwv56\" (UniqueName: \"kubernetes.io/projected/6b6fa6e5-895c-469e-8854-d5dfffec507c-kube-api-access-dwv56\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451742 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xtj2j\" (UniqueName: \"kubernetes.io/projected/0a75565a-dc09-41c9-92a3-0f3bab87860f-kube-api-access-xtj2j\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451767 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-script-lib\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451792 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-system-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-etc-kubernetes\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3cafe0e2-688f-4634-9fe6-a039e8afc21b-hosts-file\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-iptables-alerter-script\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451862 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-modprobe-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-cni-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451771 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-modprobe-d\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.452687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451743 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-run\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451905 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-systemd\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451922 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-log-socket\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451929 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-sys\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-tmp\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451965 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-var-lib-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451948 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-binary-copy\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.451975 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vwzqw\" (UniqueName: \"kubernetes.io/projected/075d2005-422a-4daa-82bb-e05e695476c0-kube-api-access-vwzqw\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452005 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-systemd\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452005 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6396047c-2712-4f87-b933-b00ad41b5d9c-konnectivity-ca\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452027 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-ovn\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452071 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452009 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-sys\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452115 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-netns\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452151 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.453283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452176 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-daemon-config\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452177 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-registration-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452207 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-netns\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452216 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kubelet-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-registration-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452231 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-host\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452259 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-config\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452260 2566 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452274 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-host\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452282 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-cni-binary-copy\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452335 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-host-slash\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452382 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-cnibin\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452384 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-host-slash\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-netd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452431 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v5wj\" (UniqueName: \"kubernetes.io/projected/3cafe0e2-688f-4634-9fe6-a039e8afc21b-kube-api-access-9v5wj\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.453946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-cnibin\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452465 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-k8s-cni-cncf-io\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452476 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-cni-netd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452495 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-netns\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452512 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/0a75565a-dc09-41c9-92a3-0f3bab87860f-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452518 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-k8s-cni-cncf-io\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452519 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-multus-certs\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452549 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-netns\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-etc-selinux\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452583 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-conf-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452591 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-run-multus-certs\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-var-lib-kubelet\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452650 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-conf-dir\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6396047c-2712-4f87-b933-b00ad41b5d9c-agent-certs\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-etc-selinux\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.454674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452687 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-system-cni-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452694 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-var-lib-kubelet\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452699 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/6b6fa6e5-895c-469e-8854-d5dfffec507c-cni-binary-copy\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452709 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452728 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-system-cni-dir\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452739 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-env-overrides\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452759 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovnkube-config\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.452768 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452762 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-hostroot\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452796 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-hostroot\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452803 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysconfig\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.452819 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:12.952802124 +0000 UTC m=+2.969173453 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452838 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysconfig\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-conf\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-etc-tuned\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452915 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-kubelet-config\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.452937 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-os-release\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453023 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-socket-dir-parent\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453031 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/075d2005-422a-4daa-82bb-e05e695476c0-etc-sysctl-conf\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453050 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-bin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453077 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-multus\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-multus-socket-dir-parent\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453084 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-env-overrides\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453041 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-os-release\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453103 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-bin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-os-release\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-host-var-lib-cni-multus\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453158 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/0a75565a-dc09-41c9-92a3-0f3bab87860f-os-release\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-systemd-units\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.455849 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453190 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-systemd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453215 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-kubelet\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453230 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453224 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-systemd-units\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453251 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovn-node-metrics-cert\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453272 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-run-systemd\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453278 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-kubelet\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cafe0e2-688f-4634-9fe6-a039e8afc21b-tmp-dir\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453286 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-host-run-ovn-kubernetes\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453338 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-grkcp\" (UniqueName: \"kubernetes.io/projected/cc95bc2d-0c04-41dc-b889-b33014478432-kube-api-access-grkcp\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453363 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-etc-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453378 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-node-log\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453393 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bgw86\" (UniqueName: \"kubernetes.io/projected/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-kube-api-access-bgw86\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453418 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-socket-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453431 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-node-log\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-device-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453466 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-sys-fs\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453471 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-etc-openvswitch\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453488 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-cnibin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453515 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453519 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-device-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453550 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3cafe0e2-688f-4634-9fe6-a039e8afc21b-tmp-dir\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453620 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-sys-fs\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453636 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/6b6fa6e5-895c-469e-8854-d5dfffec507c-cnibin\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.453639 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-socket-dir\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.455593 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-etc-tuned\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.455614 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-ovn-node-metrics-cert\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.456692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.455990 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/075d2005-422a-4daa-82bb-e05e695476c0-tmp\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.463307 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.463282 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-f4tzk\" (UniqueName: \"kubernetes.io/projected/9ff0fb0c-56b8-4d12-b30f-99e568bba580-kube-api-access-f4tzk\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.464627 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.464597 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-26r4j\" (UniqueName: \"kubernetes.io/projected/9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71-kube-api-access-26r4j\") pod \"iptables-alerter-qv4bf\" (UID: \"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71\") " pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.464782 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.464762 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwv56\" (UniqueName: \"kubernetes.io/projected/6b6fa6e5-895c-469e-8854-d5dfffec507c-kube-api-access-dwv56\") pod \"multus-nhbll\" (UID: \"6b6fa6e5-895c-469e-8854-d5dfffec507c\") " pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.465102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.465079 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwzqw\" (UniqueName: \"kubernetes.io/projected/075d2005-422a-4daa-82bb-e05e695476c0-kube-api-access-vwzqw\") pod \"tuned-xdnkr\" (UID: \"075d2005-422a-4daa-82bb-e05e695476c0\") " pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.465170 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.465135 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v5wj\" (UniqueName: \"kubernetes.io/projected/3cafe0e2-688f-4634-9fe6-a039e8afc21b-kube-api-access-9v5wj\") pod \"node-resolver-w4t9b\" (UID: \"3cafe0e2-688f-4634-9fe6-a039e8afc21b\") " pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.465635 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.465614 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bgw86\" (UniqueName: \"kubernetes.io/projected/617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3-kube-api-access-bgw86\") pod \"ovnkube-node-z2n2r\" (UID: \"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3\") " pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.465709 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.465684 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jbfz\" (UniqueName: \"kubernetes.io/projected/4eba8859-6bb0-4bb4-8f45-aa1df5a5f505-kube-api-access-6jbfz\") pod \"aws-ebs-csi-driver-node-xt4cn\" (UID: \"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.465913 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.465894 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xtj2j\" (UniqueName: \"kubernetes.io/projected/0a75565a-dc09-41c9-92a3-0f3bab87860f-kube-api-access-xtj2j\") pod \"multus-additional-cni-plugins-cm6xf\" (UID: \"0a75565a-dc09-41c9-92a3-0f3bab87860f\") " pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.520856 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.520815 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" event={"ID":"a8eeed7395da0aa41ef27a523d3170f4","Type":"ContainerStarted","Data":"9face086bc1c644e09910125252ce3b76c820380790d72bbdc486515e1891c08"} Jul 15 16:37:12.554488 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554414 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6396047c-2712-4f87-b933-b00ad41b5d9c-agent-certs\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.554488 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-kubelet-config\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.554637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554496 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-grkcp\" (UniqueName: \"kubernetes.io/projected/cc95bc2d-0c04-41dc-b889-b33014478432-kube-api-access-grkcp\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.554637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:12.554637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554552 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-dbus\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.554637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554576 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.554637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-kubelet-config\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554647 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/e38e6b54-e6dd-46e2-820a-347c749cb0b9-dbus\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554638 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc95bc2d-0c04-41dc-b889-b33014478432-host\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554690 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc95bc2d-0c04-41dc-b889-b33014478432-serviceca\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554695 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/cc95bc2d-0c04-41dc-b889-b33014478432-host\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.554722 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.554740 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6396047c-2712-4f87-b933-b00ad41b5d9c-konnectivity-ca\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.554880 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.554857 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:13.05483816 +0000 UTC m=+3.071209491 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:12.555216 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.555196 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/cc95bc2d-0c04-41dc-b889-b33014478432-serviceca\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.555278 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.555265 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/6396047c-2712-4f87-b933-b00ad41b5d9c-konnectivity-ca\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.556591 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.556576 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/6396047c-2712-4f87-b933-b00ad41b5d9c-agent-certs\") pod \"konnectivity-agent-bjvfb\" (UID: \"6396047c-2712-4f87-b933-b00ad41b5d9c\") " pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.562321 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.562300 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:12.562321 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.562320 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:12.562476 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.562344 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:12.562476 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.562397 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:13.062384534 +0000 UTC m=+3.078755890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:12.564215 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.564195 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-grkcp\" (UniqueName: \"kubernetes.io/projected/cc95bc2d-0c04-41dc-b889-b33014478432-kube-api-access-grkcp\") pod \"node-ca-g6ldc\" (UID: \"cc95bc2d-0c04-41dc-b889-b33014478432\") " pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.640974 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.640943 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" Jul 15 16:37:12.648431 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.648406 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" Jul 15 16:37:12.648613 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.648592 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0a75565a_dc09_41c9_92a3_0f3bab87860f.slice/crio-bad09c7cbde7e74ac2a0947a4e1cf674afe867d670ed7ffc15a8c3eb7cc96bda WatchSource:0}: Error finding container bad09c7cbde7e74ac2a0947a4e1cf674afe867d670ed7ffc15a8c3eb7cc96bda: Status 404 returned error can't find the container with id bad09c7cbde7e74ac2a0947a4e1cf674afe867d670ed7ffc15a8c3eb7cc96bda Jul 15 16:37:12.655900 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.655874 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-nhbll" Jul 15 16:37:12.656110 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.656086 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4eba8859_6bb0_4bb4_8f45_aa1df5a5f505.slice/crio-2a96ba24601469df29d45c291421b97de0b07bb36f26a3f93f0d25ba6b71ea50 WatchSource:0}: Error finding container 2a96ba24601469df29d45c291421b97de0b07bb36f26a3f93f0d25ba6b71ea50: Status 404 returned error can't find the container with id 2a96ba24601469df29d45c291421b97de0b07bb36f26a3f93f0d25ba6b71ea50 Jul 15 16:37:12.659905 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.659879 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-w4t9b" Jul 15 16:37:12.667300 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.667280 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:12.667766 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.667741 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6b6fa6e5_895c_469e_8854_d5dfffec507c.slice/crio-f8fb9413fa52e0f76a41cc794945a2ccd0deba21bed577e0edb91c78977c1f53 WatchSource:0}: Error finding container f8fb9413fa52e0f76a41cc794945a2ccd0deba21bed577e0edb91c78977c1f53: Status 404 returned error can't find the container with id f8fb9413fa52e0f76a41cc794945a2ccd0deba21bed577e0edb91c78977c1f53 Jul 15 16:37:12.671906 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.671882 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3cafe0e2_688f_4634_9fe6_a039e8afc21b.slice/crio-eff8f92e2d8c8f1c13a3a558a4ab5f380442e1c1357c4542cfec230616d68c4f WatchSource:0}: Error finding container eff8f92e2d8c8f1c13a3a558a4ab5f380442e1c1357c4542cfec230616d68c4f: Status 404 returned error can't find the container with id eff8f92e2d8c8f1c13a3a558a4ab5f380442e1c1357c4542cfec230616d68c4f Jul 15 16:37:12.673489 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.673467 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" Jul 15 16:37:12.677807 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.677784 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod617c55d3_8cef_4dc0_8b1a_9e81b4ae8bb3.slice/crio-2318caf2f08aaf73aa59ce0eb61e443e9968548cec5c1128208632459e8b4d34 WatchSource:0}: Error finding container 2318caf2f08aaf73aa59ce0eb61e443e9968548cec5c1128208632459e8b4d34: Status 404 returned error can't find the container with id 2318caf2f08aaf73aa59ce0eb61e443e9968548cec5c1128208632459e8b4d34 Jul 15 16:37:12.683999 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.683979 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-qv4bf" Jul 15 16:37:12.686008 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.685868 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod075d2005_422a_4daa_82bb_e05e695476c0.slice/crio-166e619c85816bbad570142a547d29df293804497754d94af1638e504fd6559d WatchSource:0}: Error finding container 166e619c85816bbad570142a547d29df293804497754d94af1638e504fd6559d: Status 404 returned error can't find the container with id 166e619c85816bbad570142a547d29df293804497754d94af1638e504fd6559d Jul 15 16:37:12.688202 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.688177 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:12.692550 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.692408 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-g6ldc" Jul 15 16:37:12.692770 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.692745 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ea6eb6d_5f09_443f_9d7c_e6ad2f7c4f71.slice/crio-af7e12acbb5de33cff52ca0690e02e5632fa382f54936f70807c7ae121bd71ba WatchSource:0}: Error finding container af7e12acbb5de33cff52ca0690e02e5632fa382f54936f70807c7ae121bd71ba: Status 404 returned error can't find the container with id af7e12acbb5de33cff52ca0690e02e5632fa382f54936f70807c7ae121bd71ba Jul 15 16:37:12.700340 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.700306 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6396047c_2712_4f87_b933_b00ad41b5d9c.slice/crio-801aa7913642c4e2592e5c5f59e55d138c97560c49d7d69ee185df1565d88aa6 WatchSource:0}: Error finding container 801aa7913642c4e2592e5c5f59e55d138c97560c49d7d69ee185df1565d88aa6: Status 404 returned error can't find the container with id 801aa7913642c4e2592e5c5f59e55d138c97560c49d7d69ee185df1565d88aa6 Jul 15 16:37:12.705089 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.704971 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcc95bc2d_0c04_41dc_b889_b33014478432.slice/crio-a83bcb5e9176d735b43586da1f18f683e86f1e8958b2e6e05ad7bd6979c04198 WatchSource:0}: Error finding container a83bcb5e9176d735b43586da1f18f683e86f1e8958b2e6e05ad7bd6979c04198: Status 404 returned error can't find the container with id a83bcb5e9176d735b43586da1f18f683e86f1e8958b2e6e05ad7bd6979c04198 Jul 15 16:37:12.755404 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:12.755371 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5033fd10d4e085ca52b1cf7265e8f08.slice/crio-c264a6914dd1e5c294d78b1a83cf983358b4adc1af67705f63cf3194806b4147 WatchSource:0}: Error finding container c264a6914dd1e5c294d78b1a83cf983358b4adc1af67705f63cf3194806b4147: Status 404 returned error can't find the container with id c264a6914dd1e5c294d78b1a83cf983358b4adc1af67705f63cf3194806b4147 Jul 15 16:37:12.850975 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.850893 2566 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 15 16:37:12.956929 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:12.956887 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:12.957081 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.957039 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:12.957145 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:12.957104 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:13.957085033 +0000 UTC m=+3.973456371 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:13.057831 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.057802 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:13.057976 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.057916 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:13.057976 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.057961 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:14.057948191 +0000 UTC m=+4.074319519 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:13.159149 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.159064 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:13.159310 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.159264 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:13.159310 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.159288 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:13.159310 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.159300 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:13.159510 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.159380 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:14.159360233 +0000 UTC m=+4.175731583 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:13.394318 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.394243 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:32:12 +0000 UTC" deadline="2028-04-04 04:47:13.449558932 +0000 UTC" Jul 15 16:37:13.394318 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.394315 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15084h10m0.055248061s" Jul 15 16:37:13.517600 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.517532 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:13.518352 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.517643 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:13.525151 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.525052 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"2318caf2f08aaf73aa59ce0eb61e443e9968548cec5c1128208632459e8b4d34"} Jul 15 16:37:13.526887 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.526859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhbll" event={"ID":"6b6fa6e5-895c-469e-8854-d5dfffec507c","Type":"ContainerStarted","Data":"f8fb9413fa52e0f76a41cc794945a2ccd0deba21bed577e0edb91c78977c1f53"} Jul 15 16:37:13.528286 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.528225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4t9b" event={"ID":"3cafe0e2-688f-4634-9fe6-a039e8afc21b","Type":"ContainerStarted","Data":"eff8f92e2d8c8f1c13a3a558a4ab5f380442e1c1357c4542cfec230616d68c4f"} Jul 15 16:37:13.529583 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.529558 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" event={"ID":"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505","Type":"ContainerStarted","Data":"2a96ba24601469df29d45c291421b97de0b07bb36f26a3f93f0d25ba6b71ea50"} Jul 15 16:37:13.532370 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.532289 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerStarted","Data":"bad09c7cbde7e74ac2a0947a4e1cf674afe867d670ed7ffc15a8c3eb7cc96bda"} Jul 15 16:37:13.534216 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.534163 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" event={"ID":"e5033fd10d4e085ca52b1cf7265e8f08","Type":"ContainerStarted","Data":"c264a6914dd1e5c294d78b1a83cf983358b4adc1af67705f63cf3194806b4147"} Jul 15 16:37:13.535823 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.535800 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6ldc" event={"ID":"cc95bc2d-0c04-41dc-b889-b33014478432","Type":"ContainerStarted","Data":"a83bcb5e9176d735b43586da1f18f683e86f1e8958b2e6e05ad7bd6979c04198"} Jul 15 16:37:13.538463 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.538438 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bjvfb" event={"ID":"6396047c-2712-4f87-b933-b00ad41b5d9c","Type":"ContainerStarted","Data":"801aa7913642c4e2592e5c5f59e55d138c97560c49d7d69ee185df1565d88aa6"} Jul 15 16:37:13.540165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.540139 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qv4bf" event={"ID":"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71","Type":"ContainerStarted","Data":"af7e12acbb5de33cff52ca0690e02e5632fa382f54936f70807c7ae121bd71ba"} Jul 15 16:37:13.542409 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.542386 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" event={"ID":"075d2005-422a-4daa-82bb-e05e695476c0","Type":"ContainerStarted","Data":"166e619c85816bbad570142a547d29df293804497754d94af1638e504fd6559d"} Jul 15 16:37:13.967249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:13.967173 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:13.967419 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.967367 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:13.967461 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:13.967423 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:15.967406169 +0000 UTC m=+5.983777504 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:14.068415 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.067852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:14.068415 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.067995 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:14.068415 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.068053 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:16.068035986 +0000 UTC m=+6.084407320 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:14.168502 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.168464 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:14.168654 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.168630 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:14.168654 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.168652 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:14.168733 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.168666 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:14.168733 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.168725 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:16.168707325 +0000 UTC m=+6.185078665 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:14.394587 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.394492 2566 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-07-14 16:32:12 +0000 UTC" deadline="2028-03-19 04:23:22.74706983 +0000 UTC" Jul 15 16:37:14.394587 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.394541 2566 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14699h46m8.352534153s" Jul 15 16:37:14.520417 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.519770 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:14.520417 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.519901 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:14.520417 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:14.520259 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:14.520417 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:14.520359 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:15.518233 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:15.517728 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:15.518233 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:15.517863 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:15.983553 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:15.983463 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:15.983984 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:15.983640 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:15.983984 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:15.983726 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:19.983704301 +0000 UTC m=+10.000075648 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:16.084035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:16.083996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:16.084249 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.084198 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:16.084321 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.084294 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:20.0842603 +0000 UTC m=+10.100631630 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:16.184819 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:16.184783 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:16.185020 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.184993 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:16.185020 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.185026 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:16.185190 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.185040 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:16.185190 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.185106 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:20.185085941 +0000 UTC m=+10.201457275 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:16.518528 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:16.518431 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:16.518528 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:16.518437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:16.518745 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.518556 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:16.518745 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:16.518654 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:17.518211 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:17.518153 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:17.518694 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:17.518301 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:18.521127 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:18.521088 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:18.521577 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:18.521221 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:18.521664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:18.521647 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:18.521766 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:18.521745 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:19.518305 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.517967 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:19.518455 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:19.518317 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:19.557092 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.556989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" event={"ID":"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505","Type":"ContainerStarted","Data":"a62a526f0c71023c3cb80555e6a9d05fb51efae20e67743f3dd1bfe42c435a44"} Jul 15 16:37:19.560151 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.560071 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerStarted","Data":"06cb82215e7db9e84c8127a8de691c09e5564ae75f2b9ecca2574fb8e8ae26ca"} Jul 15 16:37:19.565153 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.565118 2566 generic.go:356] "Generic (PLEG): container finished" podID="e5033fd10d4e085ca52b1cf7265e8f08" containerID="e7d24d58a2fb9c3d7d1c1bfb8450ea6914b11e0993f4fb97064929d7a111dcf6" exitCode=0 Jul 15 16:37:19.565255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.565243 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" event={"ID":"e5033fd10d4e085ca52b1cf7265e8f08","Type":"ContainerDied","Data":"e7d24d58a2fb9c3d7d1c1bfb8450ea6914b11e0993f4fb97064929d7a111dcf6"} Jul 15 16:37:19.573038 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.572622 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-g6ldc" event={"ID":"cc95bc2d-0c04-41dc-b889-b33014478432","Type":"ContainerStarted","Data":"150d5d19578a0b2ca8088e395ee0ddc5100937d321cfb38685eefecde70943bb"} Jul 15 16:37:19.576431 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.575959 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-bjvfb" event={"ID":"6396047c-2712-4f87-b933-b00ad41b5d9c","Type":"ContainerStarted","Data":"e24ca4e02de5a7e90d9c3749e7a0c118eb02137f7d3692b0b22a4746ed2dacc4"} Jul 15 16:37:19.579268 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.578921 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" event={"ID":"a8eeed7395da0aa41ef27a523d3170f4","Type":"ContainerStarted","Data":"aad1edf1f1084ff62080275da26149c4204aa2138be91d65da72d6c9a2404ef3"} Jul 15 16:37:19.581172 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.581148 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-w4t9b" event={"ID":"3cafe0e2-688f-4634-9fe6-a039e8afc21b","Type":"ContainerStarted","Data":"9c7a56fb2140135d6d8ce7801323394e6837ae7260cd28451f1987867b8835ce"} Jul 15 16:37:19.620624 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.620562 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-w4t9b" podStartSLOduration=3.217635407 podStartE2EDuration="9.620543718s" totalImagesPullingTime="6.402908311s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:19.603920199 +0000 UTC m=+9.620291553" watchObservedRunningTime="2026-07-15 16:37:19.620543718 +0000 UTC m=+9.636915068" Jul 15 16:37:19.639225 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.639172 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-g6ldc" podStartSLOduration=3.2725283530000002 podStartE2EDuration="9.639151391s" totalImagesPullingTime="6.366623038s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:19.638819878 +0000 UTC m=+9.655191230" watchObservedRunningTime="2026-07-15 16:37:19.639151391 +0000 UTC m=+9.655522743" Jul 15 16:37:19.639784 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.639744 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-bjvfb" podStartSLOduration=3.276886381 podStartE2EDuration="9.639729348s" totalImagesPullingTime="6.362842967s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:19.62156179 +0000 UTC m=+9.637933142" watchObservedRunningTime="2026-07-15 16:37:19.639729348 +0000 UTC m=+9.656100701" Jul 15 16:37:19.675822 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:19.675713 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-140.ec2.internal" podStartSLOduration=8.675696396 podStartE2EDuration="8.675696396s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:19.675441521 +0000 UTC m=+9.691812876" watchObservedRunningTime="2026-07-15 16:37:19.675696396 +0000 UTC m=+9.692067750" Jul 15 16:37:20.019141 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.019104 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:20.019314 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.019283 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:20.019401 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.019382 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:28.019361798 +0000 UTC m=+18.035733141 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:20.120544 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.120505 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:20.120726 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.120676 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:20.120849 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.120741 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:28.120722762 +0000 UTC m=+18.137094109 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:20.222361 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.221736 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:20.222361 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.221915 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:20.222361 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.221938 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:20.222361 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.221951 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:20.222361 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.222011 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:28.221992149 +0000 UTC m=+18.238363480 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:20.519027 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.518370 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:20.519027 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.518489 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:20.519027 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.518857 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:20.519027 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:20.518946 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:20.586061 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.586024 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="06cb82215e7db9e84c8127a8de691c09e5564ae75f2b9ecca2574fb8e8ae26ca" exitCode=0 Jul 15 16:37:20.586564 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.586115 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"06cb82215e7db9e84c8127a8de691c09e5564ae75f2b9ecca2574fb8e8ae26ca"} Jul 15 16:37:20.590594 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.590570 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" event={"ID":"e5033fd10d4e085ca52b1cf7265e8f08","Type":"ContainerStarted","Data":"b7d3d28162fff2d004bdc47562318893551662e7c06819c754e0889cf432731e"} Jul 15 16:37:20.632678 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:20.632621 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-140.ec2.internal" podStartSLOduration=9.632601501 podStartE2EDuration="9.632601501s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:11 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:20.631930445 +0000 UTC m=+10.648301797" watchObservedRunningTime="2026-07-15 16:37:20.632601501 +0000 UTC m=+10.648972853" Jul 15 16:37:21.518186 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:21.518151 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:21.518383 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:21.518278 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:21.594861 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:21.594825 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-qv4bf" event={"ID":"9ea6eb6d-5f09-443f-9d7c-e6ad2f7c4f71","Type":"ContainerStarted","Data":"cc24c8b55bad72786103944597e2ba93df9a12301e8a772ed3e868bcce786a02"} Jul 15 16:37:22.517995 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:22.517962 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:22.518159 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:22.518087 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:22.518159 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:22.518107 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:22.518280 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:22.518201 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:23.518263 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:23.518036 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:23.518680 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:23.518373 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:24.246560 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:24.246518 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:24.247119 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:24.247100 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:24.262662 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:24.262618 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-qv4bf" podStartSLOduration=7.835508953 podStartE2EDuration="14.262604447s" totalImagesPullingTime="6.427095494s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:21.612478187 +0000 UTC m=+11.628849538" watchObservedRunningTime="2026-07-15 16:37:24.262604447 +0000 UTC m=+14.278975797" Jul 15 16:37:24.518044 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:24.517947 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:24.518206 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:24.518089 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:24.518206 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:24.518144 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:24.518289 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:24.518249 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:25.517209 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:25.517176 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:25.517411 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:25.517285 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:26.517857 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:26.517822 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:26.518444 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:26.517947 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:26.518444 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:26.517979 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:26.518444 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:26.518068 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:27.517416 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:27.517374 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:27.517657 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:27.517532 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:27.824299 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:27.824220 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:27.919704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:27.919655 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:28.078251 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.078174 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:28.078425 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.078359 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:28.078479 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.078428 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:44.078406475 +0000 UTC m=+34.094777804 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:28.128016 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.127954 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:28.179430 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.179385 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:28.179584 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.179518 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:28.179584 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.179576 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:44.179559123 +0000 UTC m=+34.195930452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:28.206532 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.206466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:28.280167 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.280132 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:28.280394 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.280306 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:28.280394 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.280345 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:28.280394 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.280358 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:28.280562 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.280423 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:37:44.280404927 +0000 UTC m=+34.296776260 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:28.311251 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.311209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:28.408409 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.408313 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:28.517419 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.517388 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:28.517602 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:28.517435 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:28.517602 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.517542 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:28.517712 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:28.517667 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:29.152355 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:29.152290 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:29.230300 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:29.230197 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:29.517757 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:29.517724 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:29.517923 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:29.517857 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:29.520836 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:29.520808 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:29.621453 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:29.621413 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:30.203075 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.203041 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:30.203545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.203171 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 16:37:30.203850 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.203833 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-bjvfb" Jul 15 16:37:30.273649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.273614 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:30.518142 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.518116 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:30.519078 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:30.518681 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:30.519078 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:30.518714 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:30.519078 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:30.518846 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:31.517276 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:31.517248 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:31.517734 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:31.517355 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:31.883951 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:31.883877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:31.957443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:31.957402 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:32.517669 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:32.517641 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:32.518167 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:32.517765 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:32.518167 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:32.517812 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:32.518167 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:32.517928 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:33.102402 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:33.102368 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:33.517986 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:33.517954 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:33.518529 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:33.518086 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:34.087113 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.086551 2566 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 15 16:37:34.405275 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.405073 2566 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-15T16:37:34.08678791Z","UUID":"c8fb166e-7230-4a9f-9091-0dd424e3bd56","Handler":null,"Name":"","Endpoint":""} Jul 15 16:37:34.406286 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.406270 2566 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 15 16:37:34.406379 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.406302 2566 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 15 16:37:34.517590 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.517563 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:34.517702 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:34.517666 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:34.517759 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.517725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:34.517829 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:34.517811 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:34.622817 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.622793 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:37:34.623344 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623308 2566 generic.go:356] "Generic (PLEG): container finished" podID="617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3" containerID="f5bc74df84d0a59702500eb61d772ea9aeebb7465899d8e8644a1c09823021f7" exitCode=1 Jul 15 16:37:34.623414 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623359 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"8af1c24917d1486ae609cd67fb93e806136efca038033eb7d4f34021776d3e9c"} Jul 15 16:37:34.623468 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623435 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"53d5c218ad1185eb7b8c52f4df07c70c46fee84a25f08180eee0b93a907bbfa9"} Jul 15 16:37:34.623527 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623477 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"62b3fa84dbd72d6db7fea8a9bc314057bec656d42bcd4e329100b5912eed9692"} Jul 15 16:37:34.623527 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623514 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"ba944fc1da33c69a3cc75223fc020192fe072e067a42286cf7673d1cb0b7828c"} Jul 15 16:37:34.623607 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623540 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerDied","Data":"f5bc74df84d0a59702500eb61d772ea9aeebb7465899d8e8644a1c09823021f7"} Jul 15 16:37:34.623607 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.623581 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"e4818088e54f6492db91ac009b683d50b0df4cc4a67aea9477c1108da6af131d"} Jul 15 16:37:34.624818 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.624790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-nhbll" event={"ID":"6b6fa6e5-895c-469e-8854-d5dfffec507c","Type":"ContainerStarted","Data":"e079fb252b9efb267a8560588dad7aea997f14730beb0abe6c2889ad8f8f7405"} Jul 15 16:37:34.626620 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.626597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" event={"ID":"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505","Type":"ContainerStarted","Data":"141b96ca73bd98ba525f080f1979938449dddadd6b339744d6f9a4322252d715"} Jul 15 16:37:34.628476 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.628455 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="32c0248ca8bd5e27619cb4af32902562860f6e25cb01fee93ee52851e9ec4605" exitCode=0 Jul 15 16:37:34.628568 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.628547 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"32c0248ca8bd5e27619cb4af32902562860f6e25cb01fee93ee52851e9ec4605"} Jul 15 16:37:34.630044 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.630014 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" event={"ID":"075d2005-422a-4daa-82bb-e05e695476c0","Type":"ContainerStarted","Data":"d0a153ec5d8ed807457899bac9aeb3696e50eada60d0d1a54783b41bb63e5bb7"} Jul 15 16:37:34.643825 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.643786 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-nhbll" podStartSLOduration=3.402043307 podStartE2EDuration="24.643773731s" totalImagesPullingTime="21.241730424s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:34.643293507 +0000 UTC m=+24.659664859" watchObservedRunningTime="2026-07-15 16:37:34.643773731 +0000 UTC m=+24.660145082" Jul 15 16:37:34.684132 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.684092 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-xdnkr" podStartSLOduration=3.7036136170000002 podStartE2EDuration="24.68407889s" totalImagesPullingTime="20.980465273s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:34.68390094 +0000 UTC m=+24.700272292" watchObservedRunningTime="2026-07-15 16:37:34.68407889 +0000 UTC m=+24.700450241" Jul 15 16:37:34.765344 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.765301 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:34.866013 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:34.865981 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:35.517784 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:35.517758 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:35.518022 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:35.517854 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:36.517936 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:36.517903 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:36.518497 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:36.517903 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:36.518497 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:36.518009 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:36.518497 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:36.518095 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:36.639271 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:36.639248 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="b7b9085e5412c70f1be83acfe28e386ac44565f9cde7831f5fc57c31ba78c929" exitCode=0 Jul 15 16:37:36.639381 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:36.639291 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"b7b9085e5412c70f1be83acfe28e386ac44565f9cde7831f5fc57c31ba78c929"} Jul 15 16:37:37.518049 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:37.518021 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:37.518555 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:37.518124 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:37.643954 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:37.643926 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:37:37.644278 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:37.644257 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"898d753660bd444e00afedd1dd2a0f295da40ba7bd2924fa9eeed25317fc7500"} Jul 15 16:37:37.647935 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:37.647912 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" event={"ID":"4eba8859-6bb0-4bb4-8f45-aa1df5a5f505","Type":"ContainerStarted","Data":"3a6a5d8930bc8eb106ec9c374a519159b730ce4a10e0e6d7a86747b0af1ace27"} Jul 15 16:37:37.667763 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:37.667720 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-xt4cn" podStartSLOduration=4.309543375 podStartE2EDuration="27.667705583s" totalImagesPullingTime="23.358162208s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:37.66706533 +0000 UTC m=+27.683436681" watchObservedRunningTime="2026-07-15 16:37:37.667705583 +0000 UTC m=+27.684076933" Jul 15 16:37:38.263680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.263647 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:38.364233 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.364203 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:38.518132 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.518098 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:38.518542 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:38.518226 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:38.518542 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.518277 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:38.518542 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:38.518441 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:38.652546 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.652517 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="2f6bbafe5dd8938bf05721dafdf3c10a86969f21f5ae0cb86217e1e4a5d8e5c3" exitCode=0 Jul 15 16:37:38.652693 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:38.652597 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"2f6bbafe5dd8938bf05721dafdf3c10a86969f21f5ae0cb86217e1e4a5d8e5c3"} Jul 15 16:37:39.518104 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.517923 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:39.518272 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:39.518188 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:39.658702 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.658672 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:37:39.659176 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.659136 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"54a1d3adf8dbfbb4ada5eb5d97b43c2c68f11d86952a9ec599b52a58f9dfdadb"} Jul 15 16:37:39.659506 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.659476 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:39.659602 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.659558 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:39.659650 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.659631 2566 scope.go:117] "RemoveContainer" containerID="f5bc74df84d0a59702500eb61d772ea9aeebb7465899d8e8644a1c09823021f7" Jul 15 16:37:39.677983 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.677900 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:39.681010 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:39.680944 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:40.103754 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.103676 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:40.204173 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.204143 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:40.518405 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.518365 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:40.518823 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:40.518476 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:40.518890 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.518830 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:40.518970 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:40.518946 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:40.667486 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.667456 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:37:40.668123 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.668095 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" event={"ID":"617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3","Type":"ContainerStarted","Data":"afbde4df92359c478d4ea5f67972dc24c6f42423642ce234fda285adf92574da"} Jul 15 16:37:40.668237 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.668186 2566 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jul 15 16:37:40.700955 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:40.700898 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" podStartSLOduration=9.470459664 podStartE2EDuration="30.700874138s" totalImagesPullingTime="21.230414474s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:40.700302428 +0000 UTC m=+30.716673802" watchObservedRunningTime="2026-07-15 16:37:40.700874138 +0000 UTC m=+30.717245490" Jul 15 16:37:41.080737 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.080683 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vph5l"] Jul 15 16:37:41.080919 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.080806 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:41.080989 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:41.080920 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:41.081454 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.081432 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gv5w9"] Jul 15 16:37:41.081536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.081513 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:41.081607 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:41.081591 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:41.083473 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.083216 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rq8qh"] Jul 15 16:37:41.083473 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.083338 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:41.083473 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:41.083448 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:41.115204 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:41.115139 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:37:42.517649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:42.517614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:42.517649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:42.517653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:42.518180 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:42.517614 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:42.518180 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:42.517741 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:42.518180 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:42.517832 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:42.518180 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:42.517905 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:43.677073 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:43.676904 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="dee90a7603d5e98e33e4280295b0f236d7e9309f19eaa40432fe90da3fa41051" exitCode=0 Jul 15 16:37:43.677073 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:43.676989 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"dee90a7603d5e98e33e4280295b0f236d7e9309f19eaa40432fe90da3fa41051"} Jul 15 16:37:44.098886 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.098805 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:44.099016 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.098939 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:44.099016 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.098996 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:16.098980915 +0000 UTC m=+66.115352249 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 15 16:37:44.199193 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.199168 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:44.199338 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.199282 2566 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:44.199375 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.199343 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret podName:e38e6b54-e6dd-46e2-820a-347c749cb0b9 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:16.199314714 +0000 UTC m=+66.215686044 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret") pod "global-pull-secret-syncer-gv5w9" (UID: "e38e6b54-e6dd-46e2-820a-347c749cb0b9") : object "kube-system"/"original-pull-secret" not registered Jul 15 16:37:44.299622 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.299593 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:44.299767 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.299692 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 15 16:37:44.299767 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.299704 2566 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 15 16:37:44.299767 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.299713 2566 projected.go:196] Error preparing data for projected volume kube-api-access-96pjk for pod openshift-network-diagnostics/network-check-target-vph5l: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:44.299767 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.299756 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk podName:a103403d-b493-4f5a-8b04-04363ddc38c0 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:16.299742869 +0000 UTC m=+66.316114198 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-96pjk" (UniqueName: "kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk") pod "network-check-target-vph5l" (UID: "a103403d-b493-4f5a-8b04-04363ddc38c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 15 16:37:44.518001 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.517968 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:44.518001 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.518003 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:44.518205 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.518085 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:44.518205 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.518104 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:44.518205 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.518160 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:44.518307 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:44.518208 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:44.661459 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.661429 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:44.682559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.682532 2566 generic.go:356] "Generic (PLEG): container finished" podID="0a75565a-dc09-41c9-92a3-0f3bab87860f" containerID="1401dd3cec40cc7cebfecf165ee2e17a69d9f7a9ae0df8b3034d11c9c3f57ae9" exitCode=0 Jul 15 16:37:44.682866 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:44.682565 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerDied","Data":"1401dd3cec40cc7cebfecf165ee2e17a69d9f7a9ae0df8b3034d11c9c3f57ae9"} Jul 15 16:37:45.503824 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:45.503792 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:45.688220 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:45.688146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" event={"ID":"0a75565a-dc09-41c9-92a3-0f3bab87860f","Type":"ContainerStarted","Data":"b1e95560cac30ebcb067c3a904d8c2c8e5826269f03296731282ce8bae7fae45"} Jul 15 16:37:45.713641 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:45.713597 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-cm6xf" podStartSLOduration=9.387463315 podStartE2EDuration="35.713583249s" totalImagesPullingTime="26.326119934s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:45.712401929 +0000 UTC m=+35.728773279" watchObservedRunningTime="2026-07-15 16:37:45.713583249 +0000 UTC m=+35.729954600" Jul 15 16:37:45.839247 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:45.839212 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:45.915947 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:45.915914 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:46.517820 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:46.517793 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:46.517991 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:46.517797 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:46.517991 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:46.517890 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-gv5w9" podUID="e38e6b54-e6dd-46e2-820a-347c749cb0b9" Jul 15 16:37:46.518074 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:46.517989 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rq8qh" podUID="9ff0fb0c-56b8-4d12-b30f-99e568bba580" Jul 15 16:37:46.518074 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:46.517798 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:46.518139 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:46.518088 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vph5l" podUID="a103403d-b493-4f5a-8b04-04363ddc38c0" Jul 15 16:37:47.320947 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.320913 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:47.418306 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.418277 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:47.826995 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.826969 2566 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-140.ec2.internal" event="NodeReady" Jul 15 16:37:47.827175 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.827091 2566 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 15 16:37:47.944548 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.944511 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-ml26l"] Jul 15 16:37:47.978631 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.978606 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-m7z8z"] Jul 15 16:37:47.978760 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.978725 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:47.983295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.983115 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hj6qd\"" Jul 15 16:37:47.983432 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.983239 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 15 16:37:47.983432 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:47.983312 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 15 16:37:48.002491 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.002469 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m7z8z"] Jul 15 16:37:48.002491 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.002495 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ml26l"] Jul 15 16:37:48.002615 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.002561 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.005264 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.005248 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 15 16:37:48.005469 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.005456 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-tqn7q\"" Jul 15 16:37:48.005766 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.005752 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 15 16:37:48.005833 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.005752 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 15 16:37:48.078886 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.078820 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:48.128367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128342 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280bfa80-5182-48de-8e1e-a4dae02247fd-tmp-dir\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.128487 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128379 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kn7lm\" (UniqueName: \"kubernetes.io/projected/280bfa80-5182-48de-8e1e-a4dae02247fd-kube-api-access-kn7lm\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.128487 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128421 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d031ba50-1ba1-46c5-9ab7-28268c46b659-cert\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.128487 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128468 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.128487 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128483 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hncbh\" (UniqueName: \"kubernetes.io/projected/d031ba50-1ba1-46c5-9ab7-28268c46b659-kube-api-access-hncbh\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.128612 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.128501 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/280bfa80-5182-48de-8e1e-a4dae02247fd-config-volume\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.149986 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.149956 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:48.229358 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229309 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.229358 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229353 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hncbh\" (UniqueName: \"kubernetes.io/projected/d031ba50-1ba1-46c5-9ab7-28268c46b659-kube-api-access-hncbh\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.229517 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229380 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/280bfa80-5182-48de-8e1e-a4dae02247fd-config-volume\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.229517 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229413 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280bfa80-5182-48de-8e1e-a4dae02247fd-tmp-dir\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.229517 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229435 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kn7lm\" (UniqueName: \"kubernetes.io/projected/280bfa80-5182-48de-8e1e-a4dae02247fd-kube-api-access-kn7lm\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.229517 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:48.229435 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:37:48.229517 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:48.229502 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:37:48.729481675 +0000 UTC m=+38.745853004 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:37:48.229752 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229640 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d031ba50-1ba1-46c5-9ab7-28268c46b659-cert\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.229868 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.229847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/280bfa80-5182-48de-8e1e-a4dae02247fd-tmp-dir\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.230091 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.230074 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/280bfa80-5182-48de-8e1e-a4dae02247fd-config-volume\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.233873 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.233845 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/d031ba50-1ba1-46c5-9ab7-28268c46b659-cert\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.240052 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.240023 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hncbh\" (UniqueName: \"kubernetes.io/projected/d031ba50-1ba1-46c5-9ab7-28268c46b659-kube-api-access-hncbh\") pod \"ingress-canary-m7z8z\" (UID: \"d031ba50-1ba1-46c5-9ab7-28268c46b659\") " pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.240157 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.240139 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kn7lm\" (UniqueName: \"kubernetes.io/projected/280bfa80-5182-48de-8e1e-a4dae02247fd-kube-api-access-kn7lm\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.312407 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.312384 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-m7z8z" Jul 15 16:37:48.500768 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.500739 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-m7z8z"] Jul 15 16:37:48.504953 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:37:48.504921 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd031ba50_1ba1_46c5_9ab7_28268c46b659.slice/crio-df5bc6eb132efcb5a95bc50269221f9cec1f4249ad588e48f52ebbe75e586587 WatchSource:0}: Error finding container df5bc6eb132efcb5a95bc50269221f9cec1f4249ad588e48f52ebbe75e586587: Status 404 returned error can't find the container with id df5bc6eb132efcb5a95bc50269221f9cec1f4249ad588e48f52ebbe75e586587 Jul 15 16:37:48.517204 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.517183 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:37:48.517292 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.517187 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:37:48.517360 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.517291 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:37:48.519936 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.519916 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:37:48.521076 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.521041 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:37:48.521076 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.521058 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfbzx\"" Jul 15 16:37:48.521313 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.521086 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:37:48.521313 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.521088 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:37:48.521313 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.521143 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jvllt\"" Jul 15 16:37:48.695381 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.695304 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m7z8z" event={"ID":"d031ba50-1ba1-46c5-9ab7-28268c46b659","Type":"ContainerStarted","Data":"df5bc6eb132efcb5a95bc50269221f9cec1f4249ad588e48f52ebbe75e586587"} Jul 15 16:37:48.734723 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:48.734698 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:48.734834 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:48.734790 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:37:48.734877 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:48.734837 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:37:49.734824901 +0000 UTC m=+39.751196230 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:37:49.739985 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:49.739951 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:49.740513 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:49.740112 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:37:49.740513 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:49.740173 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:37:51.74015495 +0000 UTC m=+41.756526281 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:37:51.704931 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:51.704705 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-m7z8z" event={"ID":"d031ba50-1ba1-46c5-9ab7-28268c46b659","Type":"ContainerStarted","Data":"6cf5f869ec74ec2aa02cc7ff6591b77243b49337d08eb241a4a3e8cd0b108203"} Jul 15 16:37:51.724606 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:51.724554 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-m7z8z" podStartSLOduration=1.953744108 podStartE2EDuration="4.724536515s" totalImagesPullingTime="2.770792407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:37:51.724100065 +0000 UTC m=+41.740471439" watchObservedRunningTime="2026-07-15 16:37:51.724536515 +0000 UTC m=+41.740907867" Jul 15 16:37:51.755655 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:51.755627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:51.755791 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:51.755767 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:37:51.755831 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:51.755825 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:37:55.755810374 +0000 UTC m=+45.772181703 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:37:52.638243 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:52.638207 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:52.773807 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:52.773766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:52.972963 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:52.972938 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:53.103542 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:53.103510 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:37:55.781028 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:37:55.780996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:37:55.781415 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:55.781121 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:37:55.781415 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:37:55.781171 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:38:03.781157012 +0000 UTC m=+53.797528340 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:38:03.831711 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:03.831666 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:38:03.832241 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:03.831823 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:38:03.832241 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:03.831886 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:38:19.831870043 +0000 UTC m=+69.848241384 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:38:06.839852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:06.839814 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:12.682861 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:12.682833 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-z2n2r" Jul 15 16:38:12.852191 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:12.852156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:12.915989 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:12.915964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:12.982615 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:12.982587 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:16.109077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.109035 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:38:16.112280 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.112261 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 15 16:38:16.119489 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:16.119473 2566 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 15 16:38:16.119542 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:16.119532 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs podName:9ff0fb0c-56b8-4d12-b30f-99e568bba580 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:20.119516231 +0000 UTC m=+130.135887560 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs") pod "network-metrics-daemon-rq8qh" (UID: "9ff0fb0c-56b8-4d12-b30f-99e568bba580") : secret "metrics-daemon-secret" not found Jul 15 16:38:16.209625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.209591 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:38:16.213574 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.213555 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 15 16:38:16.223283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.223256 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/e38e6b54-e6dd-46e2-820a-347c749cb0b9-original-pull-secret\") pod \"global-pull-secret-syncer-gv5w9\" (UID: \"e38e6b54-e6dd-46e2-820a-347c749cb0b9\") " pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:38:16.310700 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.310664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:38:16.313675 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.313660 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 15 16:38:16.324053 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.324036 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 15 16:38:16.333861 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.333839 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-96pjk\" (UniqueName: \"kubernetes.io/projected/a103403d-b493-4f5a-8b04-04363ddc38c0-kube-api-access-96pjk\") pod \"network-check-target-vph5l\" (UID: \"a103403d-b493-4f5a-8b04-04363ddc38c0\") " pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:38:16.431085 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.431014 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jfbzx\"" Jul 15 16:38:16.437521 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.437501 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-gv5w9" Jul 15 16:38:16.438579 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.438565 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:38:16.565651 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.565620 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-gv5w9"] Jul 15 16:38:16.568811 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:38:16.568788 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode38e6b54_e6dd_46e2_820a_347c749cb0b9.slice/crio-1b25e5ab7a2ac373ac7a982a40331008efc82691972d936a790f0e1115bf67fe WatchSource:0}: Error finding container 1b25e5ab7a2ac373ac7a982a40331008efc82691972d936a790f0e1115bf67fe: Status 404 returned error can't find the container with id 1b25e5ab7a2ac373ac7a982a40331008efc82691972d936a790f0e1115bf67fe Jul 15 16:38:16.583105 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.583082 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vph5l"] Jul 15 16:38:16.586629 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:38:16.586606 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda103403d_b493_4f5a_8b04_04363ddc38c0.slice/crio-cebec75426e5496189dc7417f8a54a514ee1e6c73ddf1da95b867e8d68a14ffd WatchSource:0}: Error finding container cebec75426e5496189dc7417f8a54a514ee1e6c73ddf1da95b867e8d68a14ffd: Status 404 returned error can't find the container with id cebec75426e5496189dc7417f8a54a514ee1e6c73ddf1da95b867e8d68a14ffd Jul 15 16:38:16.760204 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.760172 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vph5l" event={"ID":"a103403d-b493-4f5a-8b04-04363ddc38c0","Type":"ContainerStarted","Data":"cebec75426e5496189dc7417f8a54a514ee1e6c73ddf1da95b867e8d68a14ffd"} Jul 15 16:38:16.761157 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:16.761135 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gv5w9" event={"ID":"e38e6b54-e6dd-46e2-820a-347c749cb0b9","Type":"ContainerStarted","Data":"1b25e5ab7a2ac373ac7a982a40331008efc82691972d936a790f0e1115bf67fe"} Jul 15 16:38:19.837389 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:19.837352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:38:19.837888 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:19.837484 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:38:19.837888 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:19.837547 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:38:51.837526502 +0000 UTC m=+101.853897834 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:38:21.776225 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:21.776184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vph5l" event={"ID":"a103403d-b493-4f5a-8b04-04363ddc38c0","Type":"ContainerStarted","Data":"7bfd5f129190a6e6bdd5f2a8dca30721b203ce463376f7a6835bad081c524430"} Jul 15 16:38:21.776682 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:21.776312 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:38:21.777681 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:21.777660 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-gv5w9" event={"ID":"e38e6b54-e6dd-46e2-820a-347c749cb0b9","Type":"ContainerStarted","Data":"02526c2d5d484c2510f7169233c4be5e6bf39e16fcc447540b84879a958ff3c2"} Jul 15 16:38:21.794053 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:21.794009 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vph5l" podStartSLOduration=67.36511878 podStartE2EDuration="1m11.793996864s" totalImagesPullingTime="4.428878084s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:38:21.793342289 +0000 UTC m=+71.809713637" watchObservedRunningTime="2026-07-15 16:38:21.793996864 +0000 UTC m=+71.810368214" Jul 15 16:38:21.810284 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:21.810244 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-gv5w9" podStartSLOduration=67.360315399 podStartE2EDuration="1m11.810230379s" totalImagesPullingTime="4.44991498s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:38:21.8094541 +0000 UTC m=+71.825825450" watchObservedRunningTime="2026-07-15 16:38:21.810230379 +0000 UTC m=+71.826601730" Jul 15 16:38:48.601801 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.601662 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2"] Jul 15 16:38:48.606077 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.606056 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-698f89cbfd-mss4k"] Jul 15 16:38:48.606194 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.606170 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" Jul 15 16:38:48.608513 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.608488 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.610281 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.610262 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\"" Jul 15 16:38:48.610720 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.610704 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\"" Jul 15 16:38:48.610955 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.610938 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-2vjkk\"" Jul 15 16:38:48.611026 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.610977 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jul 15 16:38:48.611514 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611494 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\"" Jul 15 16:38:48.611603 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611558 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\"" Jul 15 16:38:48.611603 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611593 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\"" Jul 15 16:38:48.611716 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611628 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jtjpv\"" Jul 15 16:38:48.611790 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611775 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jul 15 16:38:48.611790 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.611784 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\"" Jul 15 16:38:48.620634 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.620613 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2"] Jul 15 16:38:48.630196 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.630176 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-698f89cbfd-mss4k"] Jul 15 16:38:48.721489 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721462 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-stats-auth\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.721657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721498 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h7gnc\" (UniqueName: \"kubernetes.io/projected/bf3eae5b-98c4-4248-a052-44f3518fff74-kube-api-access-h7gnc\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.721657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721536 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-default-certificate\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.721657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721551 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.721657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721626 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qgxz4\" (UniqueName: \"kubernetes.io/projected/5b918283-8655-4d4c-a4bf-1fd1cf5bb107-kube-api-access-qgxz4\") pod \"volume-data-source-validator-78495597f6-xsgj2\" (UID: \"5b918283-8655-4d4c-a4bf-1fd1cf5bb107\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" Jul 15 16:38:48.721657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.721652 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.725266 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.725243 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh"] Jul 15 16:38:48.727865 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.727850 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.733236 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.733219 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 15 16:38:48.733560 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.733540 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 15 16:38:48.733636 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.733551 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jul 15 16:38:48.734785 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.734772 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 15 16:38:48.756283 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.756259 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh"] Jul 15 16:38:48.812040 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.812016 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr"] Jul 15 16:38:48.814580 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.814566 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:48.822117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822099 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-h7gnc\" (UniqueName: \"kubernetes.io/projected/bf3eae5b-98c4-4248-a052-44f3518fff74-kube-api-access-h7gnc\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.822200 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822139 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-default-certificate\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.822200 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822162 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.822200 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822181 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qgxz4\" (UniqueName: \"kubernetes.io/projected/5b918283-8655-4d4c-a4bf-1fd1cf5bb107-kube-api-access-qgxz4\") pod \"volume-data-source-validator-78495597f6-xsgj2\" (UID: \"5b918283-8655-4d4c-a4bf-1fd1cf5bb107\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" Jul 15 16:38:48.822200 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822197 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822224 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-stats-auth\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dc26ae0-d019-4aff-ac49-3a57288b577a-klusterlet-config\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822267 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fbfwr\" (UniqueName: \"kubernetes.io/projected/8dc26ae0-d019-4aff-ac49-3a57288b577a-kube-api-access-fbfwr\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:48.822290 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:49.322269717 +0000 UTC m=+99.338641066 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:48.822357 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:38:48.822383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.822366 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc26ae0-d019-4aff-ac49-3a57288b577a-tmp\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.822648 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:48.822418 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:49.32240072 +0000 UTC m=+99.338772055 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : secret "router-metrics-certs-default" not found Jul 15 16:38:48.824536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.824516 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-stats-auth\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.824617 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.824558 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-default-certificate\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.859111 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.859058 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\"" Jul 15 16:38:48.867200 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.867175 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\"" Jul 15 16:38:48.867286 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.867211 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z9vrv\"" Jul 15 16:38:48.867286 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.867187 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 15 16:38:48.870595 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.870573 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 15 16:38:48.890448 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.890426 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr"] Jul 15 16:38:48.897513 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.897494 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qgxz4\" (UniqueName: \"kubernetes.io/projected/5b918283-8655-4d4c-a4bf-1fd1cf5bb107-kube-api-access-qgxz4\") pod \"volume-data-source-validator-78495597f6-xsgj2\" (UID: \"5b918283-8655-4d4c-a4bf-1fd1cf5bb107\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" Jul 15 16:38:48.903867 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.903845 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-h7gnc\" (UniqueName: \"kubernetes.io/projected/bf3eae5b-98c4-4248-a052-44f3518fff74-kube-api-access-h7gnc\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:48.915670 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.915653 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" Jul 15 16:38:48.923706 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923685 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dc26ae0-d019-4aff-ac49-3a57288b577a-klusterlet-config\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.923781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923714 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fbfwr\" (UniqueName: \"kubernetes.io/projected/8dc26ae0-d019-4aff-ac49-3a57288b577a-kube-api-access-fbfwr\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.923781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923736 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:48.923781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:48.923781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923776 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc26ae0-d019-4aff-ac49-3a57288b577a-tmp\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.923982 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.923801 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lhbd\" (UniqueName: \"kubernetes.io/projected/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-kube-api-access-6lhbd\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:48.924147 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.924127 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8dc26ae0-d019-4aff-ac49-3a57288b577a-tmp\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.926478 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.926456 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/8dc26ae0-d019-4aff-ac49-3a57288b577a-klusterlet-config\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:48.944404 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:48.944377 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fbfwr\" (UniqueName: \"kubernetes.io/projected/8dc26ae0-d019-4aff-ac49-3a57288b577a-kube-api-access-fbfwr\") pod \"klusterlet-addon-workmgr-67dd64b4c8-z9xqh\" (UID: \"8dc26ae0-d019-4aff-ac49-3a57288b577a\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:49.024819 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.024786 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.024977 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.024834 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.024977 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.024864 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-6lhbd\" (UniqueName: \"kubernetes.io/projected/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-kube-api-access-6lhbd\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.025068 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.025000 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:49.025068 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.025066 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:49.525051438 +0000 UTC m=+99.541422772 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:49.025554 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.025536 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-telemetry-config\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.036836 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.036818 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:49.042171 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.042152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lhbd\" (UniqueName: \"kubernetes.io/projected/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-kube-api-access-6lhbd\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.043852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.043830 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2"] Jul 15 16:38:49.046880 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:38:49.046858 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b918283_8655_4d4c_a4bf_1fd1cf5bb107.slice/crio-d0be901cc95d440898617dc22d0b12fc4cd52805f8f03eee1ba50bf0e75eab64 WatchSource:0}: Error finding container d0be901cc95d440898617dc22d0b12fc4cd52805f8f03eee1ba50bf0e75eab64: Status 404 returned error can't find the container with id d0be901cc95d440898617dc22d0b12fc4cd52805f8f03eee1ba50bf0e75eab64 Jul 15 16:38:49.143829 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.143799 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:49.194853 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.194826 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh"] Jul 15 16:38:49.197837 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:38:49.197813 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8dc26ae0_d019_4aff_ac49_3a57288b577a.slice/crio-47d86004960f20cea445bd0c7afddf4dd94a0b89090560d3c52216ae41fff0f6 WatchSource:0}: Error finding container 47d86004960f20cea445bd0c7afddf4dd94a0b89090560d3c52216ae41fff0f6: Status 404 returned error can't find the container with id 47d86004960f20cea445bd0c7afddf4dd94a0b89090560d3c52216ae41fff0f6 Jul 15 16:38:49.327643 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.327612 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:49.327780 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.327695 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:49.327780 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.327760 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:38:49.327844 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.327792 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:50.327779508 +0000 UTC m=+100.344150838 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:38:49.327844 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.327807 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:50.327800007 +0000 UTC m=+100.344171336 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : secret "router-metrics-certs-default" not found Jul 15 16:38:49.371008 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.370938 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:49.529315 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.529288 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:49.529466 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.529430 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:49.529588 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:49.529490 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:50.529474677 +0000 UTC m=+100.545846007 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:49.840916 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.840873 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" event={"ID":"8dc26ae0-d019-4aff-ac49-3a57288b577a","Type":"ContainerStarted","Data":"47d86004960f20cea445bd0c7afddf4dd94a0b89090560d3c52216ae41fff0f6"} Jul 15 16:38:49.842170 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:49.842146 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" event={"ID":"5b918283-8655-4d4c-a4bf-1fd1cf5bb107","Type":"ContainerStarted","Data":"d0be901cc95d440898617dc22d0b12fc4cd52805f8f03eee1ba50bf0e75eab64"} Jul 15 16:38:50.336058 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.336020 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:50.336058 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.336062 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:50.336269 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:50.336212 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:38:50.336269 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:50.336215 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:52.336190961 +0000 UTC m=+102.352562309 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:38:50.336269 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:50.336258 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:52.336246778 +0000 UTC m=+102.352618107 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : secret "router-metrics-certs-default" not found Jul 15 16:38:50.538364 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.538311 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:50.538546 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:50.538472 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:50.538608 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:50.538548 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:52.538527757 +0000 UTC m=+102.554899101 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:50.847686 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.847637 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" event={"ID":"5b918283-8655-4d4c-a4bf-1fd1cf5bb107","Type":"ContainerStarted","Data":"ae6a505e0e34e8921624c90151ed4593a8afe05150c4beb60ed4e35b001dfc9c"} Jul 15 16:38:50.867597 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.867550 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-78495597f6-xsgj2" podStartSLOduration=1.878314106 podStartE2EDuration="2.867534642s" totalImagesPullingTime="989.220536ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:38:50.865929644 +0000 UTC m=+100.882300997" watchObservedRunningTime="2026-07-15 16:38:50.867534642 +0000 UTC m=+100.883905992" Jul 15 16:38:50.987741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:50.987703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:51.851130 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:51.851102 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:38:51.851569 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:51.851234 2566 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 15 16:38:51.851569 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:51.851288 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls podName:280bfa80-5182-48de-8e1e-a4dae02247fd nodeName:}" failed. No retries permitted until 2026-07-15 16:39:55.851272579 +0000 UTC m=+165.867643915 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls") pod "dns-default-ml26l" (UID: "280bfa80-5182-48de-8e1e-a4dae02247fd") : secret "dns-default-metrics-tls" not found Jul 15 16:38:52.353593 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.353557 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:52.353763 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.353605 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:52.353846 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.353771 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:38:52.353846 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.353772 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:56.353747862 +0000 UTC m=+106.370119202 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:38:52.353846 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.353836 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:56.353817786 +0000 UTC m=+106.370189115 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : secret "router-metrics-certs-default" not found Jul 15 16:38:52.555501 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.555468 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:52.555671 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.555619 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:52.555765 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.555679 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:56.555662747 +0000 UTC m=+106.572034090 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:52.557212 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.557176 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:52.725063 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.725031 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lczwb"] Jul 15 16:38:52.744841 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.744789 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lczwb"] Jul 15 16:38:52.744998 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.744930 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.748055 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.748033 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 15 16:38:52.748055 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.748035 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 15 16:38:52.748255 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.748096 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 15 16:38:52.749884 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.749861 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7mm5\"" Jul 15 16:38:52.750002 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.749916 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 15 16:38:52.768071 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.768044 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:52.782860 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.782833 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vph5l" Jul 15 16:38:52.858372 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.858313 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-data-volume\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.858802 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.858441 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.858802 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.858499 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-l449k\" (UniqueName: \"kubernetes.io/projected/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-api-access-l449k\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.858802 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.858534 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-crio-socket\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.858802 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.858557 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.871582 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.871555 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:52.959502 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959669 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-data-volume\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959669 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.959600 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:38:52.959669 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959642 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959803 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.959675 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:53.459644044 +0000 UTC m=+103.476015374 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : configmap "kube-rbac-proxy" not found Jul 15 16:38:52.959803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959729 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-l449k\" (UniqueName: \"kubernetes.io/projected/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-api-access-l449k\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959803 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.959752 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:38:52.959803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-crio-socket\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959803 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:52.959802 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:53.459787155 +0000 UTC m=+103.476158495 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:38:52.959996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959853 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-crio-socket\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.959996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.959964 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-data-volume\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:52.975721 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:52.975657 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-l449k\" (UniqueName: \"kubernetes.io/projected/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-api-access-l449k\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:53.463934 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:53.463910 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:53.464079 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:53.463952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:53.464079 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:53.464033 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:38:53.464079 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:53.464053 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:38:53.464180 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:53.464083 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:54.464071479 +0000 UTC m=+104.480442807 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : configmap "kube-rbac-proxy" not found Jul 15 16:38:53.464180 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:53.464102 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:54.464090013 +0000 UTC m=+104.480461342 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:38:54.471247 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.471213 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:54.471678 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.471305 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:54.471678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:54.471351 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:38:54.471678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:54.471405 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:38:54.471678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:54.471421 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:56.471405092 +0000 UTC m=+106.487776423 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : configmap "kube-rbac-proxy" not found Jul 15 16:38:54.471678 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:54.471446 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:38:56.471434623 +0000 UTC m=+106.487805952 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:38:54.683595 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.683570 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4t9b_3cafe0e2-688f-4634-9fe6-a039e8afc21b/dns-node-resolver/0.log" Jul 15 16:38:54.859756 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.859675 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" event={"ID":"8dc26ae0-d019-4aff-ac49-3a57288b577a","Type":"ContainerStarted","Data":"6a5e1a31f9fc9cbe33e28885f3dd3926c2a44ea35e8be35b0982768130cb8e62"} Jul 15 16:38:54.859919 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.859885 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:54.861447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.861425 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" Jul 15 16:38:54.878936 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.878900 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-67dd64b4c8-z9xqh" podStartSLOduration=2.28231923 podStartE2EDuration="6.878887245s" totalImagesPullingTime="4.596568015s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:38:54.878137982 +0000 UTC m=+104.894509332" watchObservedRunningTime="2026-07-15 16:38:54.878887245 +0000 UTC m=+104.895258592" Jul 15 16:38:54.998912 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:54.998885 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:55.021980 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.021951 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887"] Jul 15 16:38:55.044910 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.044889 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887"] Jul 15 16:38:55.045023 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.044998 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" Jul 15 16:38:55.047855 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.047834 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-gkkkq\"" Jul 15 16:38:55.047952 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.047834 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jul 15 16:38:55.047952 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.047874 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jul 15 16:38:55.093316 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.093292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:55.176031 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.175976 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67phs\" (UniqueName: \"kubernetes.io/projected/3347e66c-467c-4342-9d86-15e835b219c9-kube-api-access-67phs\") pod \"migrator-7f59b54f6c-mc887\" (UID: \"3347e66c-467c-4342-9d86-15e835b219c9\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" Jul 15 16:38:55.179721 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.179696 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:55.276881 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.276852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-67phs\" (UniqueName: \"kubernetes.io/projected/3347e66c-467c-4342-9d86-15e835b219c9-kube-api-access-67phs\") pod \"migrator-7f59b54f6c-mc887\" (UID: \"3347e66c-467c-4342-9d86-15e835b219c9\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" Jul 15 16:38:55.287344 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.287306 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-67phs\" (UniqueName: \"kubernetes.io/projected/3347e66c-467c-4342-9d86-15e835b219c9-kube-api-access-67phs\") pod \"migrator-7f59b54f6c-mc887\" (UID: \"3347e66c-467c-4342-9d86-15e835b219c9\") " pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" Jul 15 16:38:55.353479 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.353455 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" Jul 15 16:38:55.459187 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.459160 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:55.470097 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.470075 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887"] Jul 15 16:38:55.472602 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:38:55.472579 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3347e66c_467c_4342_9d86_15e835b219c9.slice/crio-616906cd74fdf1d08d94ad892cb3aba3db9bd368f57147626072e5071edab74f WatchSource:0}: Error finding container 616906cd74fdf1d08d94ad892cb3aba3db9bd368f57147626072e5071edab74f: Status 404 returned error can't find the container with id 616906cd74fdf1d08d94ad892cb3aba3db9bd368f57147626072e5071edab74f Jul 15 16:38:55.482560 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.482544 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6ldc_cc95bc2d-0c04-41dc-b889-b33014478432/node-ca/0.log" Jul 15 16:38:55.559240 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.559216 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:55.863140 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:55.863103 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" event={"ID":"3347e66c-467c-4342-9d86-15e835b219c9","Type":"ContainerStarted","Data":"616906cd74fdf1d08d94ad892cb3aba3db9bd368f57147626072e5071edab74f"} Jul 15 16:38:56.121558 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.121491 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:56.226233 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.226209 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:56.386434 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.386341 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:56.386434 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.386398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:38:56.386614 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.386497 2566 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found Jul 15 16:38:56.386614 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.386516 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:04.386500343 +0000 UTC m=+114.402871672 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:38:56.386614 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.386559 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:04.386547633 +0000 UTC m=+114.402918962 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : secret "router-metrics-certs-default" not found Jul 15 16:38:56.483741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.483709 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m7z8z_d031ba50-1ba1-46c5-9ab7-28268c46b659/serve-healthcheck-canary/0.log" Jul 15 16:38:56.486962 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.486918 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:56.487085 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.486976 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:38:56.487085 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.487077 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:38:56.487185 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.487098 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:38:56.487185 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.487158 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:00.487138518 +0000 UTC m=+110.503509855 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:38:56.487185 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.487180 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:00.487168514 +0000 UTC m=+110.503539850 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : configmap "kube-rbac-proxy" not found Jul 15 16:38:56.588173 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:56.588144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:38:56.588351 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.588284 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:56.588402 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:38:56.588356 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:04.58834173 +0000 UTC m=+114.604713059 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:38:57.768544 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:57.768513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:57.841817 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:57.841784 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:57.871367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:57.871338 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" event={"ID":"3347e66c-467c-4342-9d86-15e835b219c9","Type":"ContainerStarted","Data":"31a54dc4946b905a7c36bbad866a4d2e8d9d30725f7dff36f129306de5df0d84"} Jul 15 16:38:57.871498 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:57.871395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" event={"ID":"3347e66c-467c-4342-9d86-15e835b219c9","Type":"ContainerStarted","Data":"5952eda7da9417b80ca3c453fbeadb561de72f739a0b60d3c0d32d7f27673cca"} Jul 15 16:38:57.890569 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:57.890527 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-7f59b54f6c-mc887" podStartSLOduration=1.250117087 podStartE2EDuration="2.890513307s" totalImagesPullingTime="1.64039622s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:38:55 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:38:57.889557177 +0000 UTC m=+107.905928540" watchObservedRunningTime="2026-07-15 16:38:57.890513307 +0000 UTC m=+107.906884657" Jul 15 16:38:58.002280 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:58.002249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:58.072101 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:58.072036 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:58.803621 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:58.803586 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:59.479544 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:59.479513 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:59.694419 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:59.694389 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:38:59.942932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:38:59.942845 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:00.172703 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:00.172673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:00.398636 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:00.398611 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:00.515977 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:00.515952 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:00.516135 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:00.515989 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:00.516135 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:00.516072 2566 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 15 16:39:00.516135 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:00.516095 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:39:00.516135 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:00.516122 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:08.516110742 +0000 UTC m=+118.532482070 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : configmap "kube-rbac-proxy" not found Jul 15 16:39:00.516272 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:00.516152 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:08.516138479 +0000 UTC m=+118.532509807 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:39:00.600686 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:00.600658 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:02.428665 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:02.428633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:02.642894 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:02.642864 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:02.745408 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:02.745382 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:02.845175 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:02.845142 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:03.213449 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:03.213422 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:03.371046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:03.371017 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:03.526738 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:03.526657 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:03.791578 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:03.791492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:04.205015 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.204929 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:04.448161 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.448125 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:04.448349 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.448245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:04.448428 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:04.448393 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle podName:bf3eae5b-98c4-4248-a052-44f3518fff74 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:20.448374377 +0000 UTC m=+130.464745720 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle") pod "router-default-698f89cbfd-mss4k" (UID: "bf3eae5b-98c4-4248-a052-44f3518fff74") : configmap references non-existent config key: service-ca.crt Jul 15 16:39:04.450422 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.450402 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/bf3eae5b-98c4-4248-a052-44f3518fff74-metrics-certs\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:04.573186 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.573156 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:04.650061 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.650017 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:39:04.650190 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:04.650176 2566 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found Jul 15 16:39:04.650241 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:04.650231 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls podName:5bcb7a32-be70-49c9-a0eb-0131ccc90a61 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:20.65021524 +0000 UTC m=+130.666586586 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-5f7f494dd4-gfspr" (UID: "5bcb7a32-be70-49c9-a0eb-0131ccc90a61") : secret "cluster-monitoring-operator-tls" not found Jul 15 16:39:04.771446 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:04.771415 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:05.054243 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:05.054214 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:05.418479 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:05.418390 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:06.798469 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:06.798440 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:06.898891 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:06.898864 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:06.999927 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:06.999896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:07.380274 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:07.380248 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:07.571957 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:07.571928 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:07.972959 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:07.972923 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:08.581649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:08.581618 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:08.581808 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:08.581660 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:08.581808 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:08.581780 2566 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 15 16:39:08.581880 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:08.581851 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls podName:fafc9049-86af-4af5-b9ed-cd5a5ad2fc00 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:24.581835759 +0000 UTC m=+134.598207088 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lczwb" (UID: "fafc9049-86af-4af5-b9ed-cd5a5ad2fc00") : secret "insights-runtime-extractor-tls" not found Jul 15 16:39:08.582116 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:08.582098 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:08.590600 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:08.590573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:09.178508 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:09.178475 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:09.374091 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:09.374062 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:09.574425 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:09.574395 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:09.781367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:09.781318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:10.170147 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:10.170120 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:10.838112 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:10.838082 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:10.939170 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:10.939135 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.041060 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.041020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.139807 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.139733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.589797 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.589769 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.673247 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.673219 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.777354 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.777320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:11.959955 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:11.959882 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:12.127503 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:12.127470 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:12.414873 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:12.414846 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:12.558896 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:12.558866 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:12.658996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:12.658958 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:13.963397 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:13.963367 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:14.081009 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:14.080978 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:14.519154 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:14.519129 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:14.619170 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:14.619139 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:18.879996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.879964 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-6955c964f4-5tqp7"] Jul 15 16:39:18.882062 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.882048 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:18.886289 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.886266 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 15 16:39:18.887317 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.887287 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 15 16:39:18.888623 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.888601 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-rgnls\"" Jul 15 16:39:18.898704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.898685 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 15 16:39:18.900242 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.900222 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:18.913554 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.913534 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6955c964f4-5tqp7"] Jul 15 16:39:18.913808 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:18.913792 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 15 16:39:19.057371 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057336 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-certificates\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057374 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-ca-trust-extracted\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057492 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-installation-pull-secrets\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057519 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-bound-sa-token\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057543 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-image-registry-private-configuration\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057559 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qmbww\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-kube-api-access-qmbww\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057609 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-trusted-ca\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.057645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.057633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-tls\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158259 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158188 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-image-registry-private-configuration\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158259 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158218 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qmbww\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-kube-api-access-qmbww\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158265 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-trusted-ca\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158292 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-tls\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158366 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-certificates\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158398 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-ca-trust-extracted\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158631 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158475 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-installation-pull-secrets\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.158631 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.158501 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-bound-sa-token\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.159050 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.159001 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-ca-trust-extracted\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.159344 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.159305 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-certificates\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.159437 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.159349 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-trusted-ca\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.160829 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.160807 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-installation-pull-secrets\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.160912 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.160860 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-image-registry-private-configuration\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.161100 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.161083 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-registry-tls\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.168084 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.168057 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-bound-sa-token\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.168220 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.168203 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-qmbww\" (UniqueName: \"kubernetes.io/projected/91e76f29-ffd4-4dd8-be9d-bdafa83e7dec-kube-api-access-qmbww\") pod \"image-registry-6955c964f4-5tqp7\" (UID: \"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec\") " pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.191257 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.191230 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.318477 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.318451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-6955c964f4-5tqp7"] Jul 15 16:39:19.320801 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:19.320775 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod91e76f29_ffd4_4dd8_be9d_bdafa83e7dec.slice/crio-01f97f05837c32f74937da251c93e0af949ce8460dea2ff67d43a962af6760d5 WatchSource:0}: Error finding container 01f97f05837c32f74937da251c93e0af949ce8460dea2ff67d43a962af6760d5: Status 404 returned error can't find the container with id 01f97f05837c32f74937da251c93e0af949ce8460dea2ff67d43a962af6760d5 Jul 15 16:39:19.935777 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.935743 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" event={"ID":"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec","Type":"ContainerStarted","Data":"3eb5637d7755a0184cf4c7fab99c446ba2a183ca73f2fa65f369315732ea6972"} Jul 15 16:39:19.936176 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.935792 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" event={"ID":"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec","Type":"ContainerStarted","Data":"01f97f05837c32f74937da251c93e0af949ce8460dea2ff67d43a962af6760d5"} Jul 15 16:39:19.936176 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.935859 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:19.958145 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:19.958095 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podStartSLOduration=1.95808176 podStartE2EDuration="1.95808176s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:18 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:19.957867122 +0000 UTC m=+129.974238472" watchObservedRunningTime="2026-07-15 16:39:19.95808176 +0000 UTC m=+129.974453110" Jul 15 16:39:20.166523 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.166484 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:39:20.168657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.168637 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9ff0fb0c-56b8-4d12-b30f-99e568bba580-metrics-certs\") pod \"network-metrics-daemon-rq8qh\" (UID: \"9ff0fb0c-56b8-4d12-b30f-99e568bba580\") " pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:39:20.335436 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.335408 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-jvllt\"" Jul 15 16:39:20.343590 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.343572 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rq8qh" Jul 15 16:39:20.455131 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.455096 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:20.468995 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.468967 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:20.469607 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.469591 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/bf3eae5b-98c4-4248-a052-44f3518fff74-service-ca-bundle\") pod \"router-default-698f89cbfd-mss4k\" (UID: \"bf3eae5b-98c4-4248-a052-44f3518fff74\") " pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:20.476296 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.476276 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rq8qh"] Jul 15 16:39:20.479341 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:20.479304 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ff0fb0c_56b8_4d12_b30f_99e568bba580.slice/crio-56c574bf425641364d6120562b13416016c9c465638f3ff030b3dca9c7dc6c0e WatchSource:0}: Error finding container 56c574bf425641364d6120562b13416016c9c465638f3ff030b3dca9c7dc6c0e: Status 404 returned error can't find the container with id 56c574bf425641364d6120562b13416016c9c465638f3ff030b3dca9c7dc6c0e Jul 15 16:39:20.671288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.671221 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:39:20.673457 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.673432 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/5bcb7a32-be70-49c9-a0eb-0131ccc90a61-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-5f7f494dd4-gfspr\" (UID: \"5bcb7a32-be70-49c9-a0eb-0131ccc90a61\") " pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:39:20.725041 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.725022 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-jtjpv\"" Jul 15 16:39:20.732391 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.732377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:20.854396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.854337 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-698f89cbfd-mss4k"] Jul 15 16:39:20.857689 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:20.857658 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf3eae5b_98c4_4248_a052_44f3518fff74.slice/crio-b941fa7c04a0b9ecc1f1da00cbfa3ece5fc42cdb6c1decfcc39a6b5d3f83ae80 WatchSource:0}: Error finding container b941fa7c04a0b9ecc1f1da00cbfa3ece5fc42cdb6c1decfcc39a6b5d3f83ae80: Status 404 returned error can't find the container with id b941fa7c04a0b9ecc1f1da00cbfa3ece5fc42cdb6c1decfcc39a6b5d3f83ae80 Jul 15 16:39:20.924720 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.924656 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-z9vrv\"" Jul 15 16:39:20.932692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.932668 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" Jul 15 16:39:20.939971 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.939938 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rq8qh" event={"ID":"9ff0fb0c-56b8-4d12-b30f-99e568bba580","Type":"ContainerStarted","Data":"56c574bf425641364d6120562b13416016c9c465638f3ff030b3dca9c7dc6c0e"} Jul 15 16:39:20.941765 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.941735 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-698f89cbfd-mss4k" event={"ID":"bf3eae5b-98c4-4248-a052-44f3518fff74","Type":"ContainerStarted","Data":"055b59bb4aa0e76185db47effc33fd2a29d5f75b770437c9909b9349d4955837"} Jul 15 16:39:20.941853 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.941776 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-698f89cbfd-mss4k" event={"ID":"bf3eae5b-98c4-4248-a052-44f3518fff74","Type":"ContainerStarted","Data":"b941fa7c04a0b9ecc1f1da00cbfa3ece5fc42cdb6c1decfcc39a6b5d3f83ae80"} Jul 15 16:39:20.964147 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:20.964094 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/router-default-698f89cbfd-mss4k" podStartSLOduration=32.964074459 podStartE2EDuration="32.964074459s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:38:48 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:20.962406085 +0000 UTC m=+130.978777436" watchObservedRunningTime="2026-07-15 16:39:20.964074459 +0000 UTC m=+130.980445809" Jul 15 16:39:21.079098 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.079065 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr"] Jul 15 16:39:21.082108 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:21.082082 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5bcb7a32_be70_49c9_a0eb_0131ccc90a61.slice/crio-c7c921c01adf6625516ac98c3f07da3cd785016a9c5d01e0e7f66451c19f21b6 WatchSource:0}: Error finding container c7c921c01adf6625516ac98c3f07da3cd785016a9c5d01e0e7f66451c19f21b6: Status 404 returned error can't find the container with id c7c921c01adf6625516ac98c3f07da3cd785016a9c5d01e0e7f66451c19f21b6 Jul 15 16:39:21.732968 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.732931 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:21.735368 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.735345 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:21.946563 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.946522 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" event={"ID":"5bcb7a32-be70-49c9-a0eb-0131ccc90a61","Type":"ContainerStarted","Data":"c7c921c01adf6625516ac98c3f07da3cd785016a9c5d01e0e7f66451c19f21b6"} Jul 15 16:39:21.947004 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.946776 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:21.948121 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:21.948089 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-698f89cbfd-mss4k" Jul 15 16:39:23.379089 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.379054 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:23.513314 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.513282 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:23.669090 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.669059 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:23.878894 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.878772 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:23.879504 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.879486 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp"] Jul 15 16:39:23.882382 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.882368 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:23.886340 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.886311 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-tls\"" Jul 15 16:39:23.886459 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.886364 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-admission-webhook-dockercfg-ldj49\"" Jul 15 16:39:23.896827 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.896807 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp"] Jul 15 16:39:23.955342 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.955296 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rq8qh" event={"ID":"9ff0fb0c-56b8-4d12-b30f-99e568bba580","Type":"ContainerStarted","Data":"94d11d21569c7cc2686d747ed3514d6402f35c73bd2c8a09f9f16cd00a584b69"} Jul 15 16:39:23.955525 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.955369 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rq8qh" event={"ID":"9ff0fb0c-56b8-4d12-b30f-99e568bba580","Type":"ContainerStarted","Data":"1520df74b5c05aba19fc1b0a6e000746e7c1df93ca69ebc79f68c183501d72d4"} Jul 15 16:39:23.956675 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.956649 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" event={"ID":"5bcb7a32-be70-49c9-a0eb-0131ccc90a61","Type":"ContainerStarted","Data":"38f8b51bfdebf3197fce34648e525d84109e95b6c3498461e06c2c7ecfc7bcd9"} Jul 15 16:39:23.969805 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.969782 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:23.976477 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.976343 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rq8qh" podStartSLOduration=131.313130117 podStartE2EDuration="2m13.976312943s" totalImagesPullingTime="2.663182826s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:10 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:23.975889471 +0000 UTC m=+133.992260824" watchObservedRunningTime="2026-07-15 16:39:23.976312943 +0000 UTC m=+133.992684291" Jul 15 16:39:23.996863 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.996802 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-5f7f494dd4-gfspr" podStartSLOduration=33.932824515 podStartE2EDuration="35.996789458s" totalImagesPullingTime="2.063964943s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:38:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:23.995939933 +0000 UTC m=+134.012311284" watchObservedRunningTime="2026-07-15 16:39:23.996789458 +0000 UTC m=+134.013160810" Jul 15 16:39:23.998950 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:23.998924 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96b31b8a-9e99-47e0-9acf-dbec747d8d20-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qrnkp\" (UID: \"96b31b8a-9e99-47e0-9acf-dbec747d8d20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:24.099998 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.099962 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96b31b8a-9e99-47e0-9acf-dbec747d8d20-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qrnkp\" (UID: \"96b31b8a-9e99-47e0-9acf-dbec747d8d20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:24.102583 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.102541 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certificates\" (UniqueName: \"kubernetes.io/secret/96b31b8a-9e99-47e0-9acf-dbec747d8d20-tls-certificates\") pod \"prometheus-operator-admission-webhook-77d98bc786-qrnkp\" (UID: \"96b31b8a-9e99-47e0-9acf-dbec747d8d20\") " pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:24.141753 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.141680 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:24.191163 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.191137 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:24.245093 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.245064 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:24.320303 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.320271 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp"] Jul 15 16:39:24.325990 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:24.323813 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96b31b8a_9e99_47e0_9acf_dbec747d8d20.slice/crio-c464824d7ce8c3dcabd20e05d6158e12f34309a27488b885764d55ba33876871 WatchSource:0}: Error finding container c464824d7ce8c3dcabd20e05d6158e12f34309a27488b885764d55ba33876871: Status 404 returned error can't find the container with id c464824d7ce8c3dcabd20e05d6158e12f34309a27488b885764d55ba33876871 Jul 15 16:39:24.604007 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.603979 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:24.606423 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.606399 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/fafc9049-86af-4af5-b9ed-cd5a5ad2fc00-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lczwb\" (UID: \"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00\") " pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:24.734187 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.734150 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:24.858754 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.858687 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-j7mm5\"" Jul 15 16:39:24.866494 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.866436 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lczwb" Jul 15 16:39:24.907788 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.907755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:24.964687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:24.964651 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" event={"ID":"96b31b8a-9e99-47e0-9acf-dbec747d8d20","Type":"ContainerStarted","Data":"c464824d7ce8c3dcabd20e05d6158e12f34309a27488b885764d55ba33876871"} Jul 15 16:39:25.021504 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.021478 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lczwb"] Jul 15 16:39:25.024624 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:25.024591 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfafc9049_86af_4af5_b9ed_cd5a5ad2fc00.slice/crio-460479cdc2e18d20892b0c149f7b55352d57e0b234f4e63254d62229addf6dff WatchSource:0}: Error finding container 460479cdc2e18d20892b0c149f7b55352d57e0b234f4e63254d62229addf6dff: Status 404 returned error can't find the container with id 460479cdc2e18d20892b0c149f7b55352d57e0b234f4e63254d62229addf6dff Jul 15 16:39:25.192752 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.192657 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:39:25.193773 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.193733 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:25.197422 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.197400 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.201757 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.201729 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 16:39:25.202307 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.202277 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 16:39:25.202506 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.202431 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 16:39:25.202506 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.202470 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 16:39:25.202649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.202517 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 16:39:25.203659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.203132 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mtjsr\"" Jul 15 16:39:25.203659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.203427 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 16:39:25.208636 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.203917 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 16:39:25.209772 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.209749 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:39:25.212017 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.211997 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 16:39:25.265774 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.265741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:25.310031 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310003 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310031 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310039 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310078 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310096 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkf57\" (UniqueName: \"kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.310265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.310152 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411584 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411546 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411594 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411626 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411664 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411908 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411737 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkf57\" (UniqueName: \"kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411908 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411796 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.411908 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.411852 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.412462 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.412433 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.412604 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.412547 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.412866 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.412843 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.412979 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.412911 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.414621 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.414599 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.414777 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.414756 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.426247 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.426222 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkf57\" (UniqueName: \"kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57\") pod \"console-7d8b489f9c-j5sgv\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.519469 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.519437 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:25.660883 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.660853 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:39:25.664030 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:25.664006 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfd2ba17b_9c4e_4338_95d8_676068b300f8.slice/crio-beb4cb031c79daad89fd822f15ba1e7303f134254c6dbae97e2b2463a6bc5b57 WatchSource:0}: Error finding container beb4cb031c79daad89fd822f15ba1e7303f134254c6dbae97e2b2463a6bc5b57: Status 404 returned error can't find the container with id beb4cb031c79daad89fd822f15ba1e7303f134254c6dbae97e2b2463a6bc5b57 Jul 15 16:39:25.969066 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.969030 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" event={"ID":"96b31b8a-9e99-47e0-9acf-dbec747d8d20","Type":"ContainerStarted","Data":"62aec0016c9d8f6be9daa1ef4a20cdf0380f10286ea01f77fb89e5f21a121947"} Jul 15 16:39:25.969256 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.969228 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:25.970272 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.970246 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8b489f9c-j5sgv" event={"ID":"fd2ba17b-9c4e-4338-95d8-676068b300f8","Type":"ContainerStarted","Data":"beb4cb031c79daad89fd822f15ba1e7303f134254c6dbae97e2b2463a6bc5b57"} Jul 15 16:39:25.971401 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.971382 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lczwb" event={"ID":"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00","Type":"ContainerStarted","Data":"460479cdc2e18d20892b0c149f7b55352d57e0b234f4e63254d62229addf6dff"} Jul 15 16:39:25.974008 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:25.973990 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" Jul 15 16:39:26.005589 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:26.005542 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-admission-webhook-77d98bc786-qrnkp" podStartSLOduration=1.749251987 podStartE2EDuration="3.005529428s" totalImagesPullingTime="1.256277441s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:25.987419251 +0000 UTC m=+136.003790606" watchObservedRunningTime="2026-07-15 16:39:26.005529428 +0000 UTC m=+136.021900779" Jul 15 16:39:26.965164 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:26.965126 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:27.122967 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:27.122896 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:27.389795 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:27.389715 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:29.754135 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:29.754080 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:29.882376 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:29.882343 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:30.116828 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:30.116752 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:31.995337 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:31.995292 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8b489f9c-j5sgv" event={"ID":"fd2ba17b-9c4e-4338-95d8-676068b300f8","Type":"ContainerStarted","Data":"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e"} Jul 15 16:39:31.996747 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:31.996722 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lczwb" event={"ID":"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00","Type":"ContainerStarted","Data":"3f5c01feda40d4df3f617cd88caa101f0844554d37851da9752785bd59778f69"} Jul 15 16:39:32.019002 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.018955 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-7d8b489f9c-j5sgv" podStartSLOduration=1.175130253 podStartE2EDuration="7.018940728s" totalImagesPullingTime="5.843810475s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:25 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:32.017890829 +0000 UTC m=+142.034262181" watchObservedRunningTime="2026-07-15 16:39:32.018940728 +0000 UTC m=+142.035312079" Jul 15 16:39:32.152814 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.152782 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.251710 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.251633 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.375081 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.375041 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.420667 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.420639 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d"] Jul 15 16:39:32.424340 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.424310 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.428911 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.428889 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\"" Jul 15 16:39:32.429300 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.429277 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\"" Jul 15 16:39:32.430065 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.430044 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-r59r9\"" Jul 15 16:39:32.430128 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.430051 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 15 16:39:32.441170 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.441148 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d"] Jul 15 16:39:32.467906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.467883 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.468012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.467918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6766722-4419-4487-82cd-f231339815a1-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.468012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.467940 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.468085 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.468018 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmfr8\" (UniqueName: \"kubernetes.io/projected/f6766722-4419-4487-82cd-f231339815a1-kube-api-access-xmfr8\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.498609 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.498583 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cxnqd"] Jul 15 16:39:32.501432 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.501408 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.501886 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.501839 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.505284 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.505265 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 15 16:39:32.505577 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.505561 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 15 16:39:32.505679 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.505664 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 15 16:39:32.505732 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.505680 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-jgbnv\"" Jul 15 16:39:32.568302 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xmfr8\" (UniqueName: \"kubernetes.io/projected/f6766722-4419-4487-82cd-f231339815a1-kube-api-access-xmfr8\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.568445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568306 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568343 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-sys\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568375 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.568445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568420 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-thgl2\" (UniqueName: \"kubernetes.io/projected/f1e906a7-ea5d-4393-9573-5461c28b2681-kube-api-access-thgl2\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568466 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-textfile\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568491 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6766722-4419-4487-82cd-f231339815a1-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568558 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-root\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568592 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.568649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568629 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-wtmp\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568905 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568655 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-metrics-client-ca\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568905 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:32.568676 2566 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found Jul 15 16:39:32.568905 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.568686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.568905 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:32.568739 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls podName:f6766722-4419-4487-82cd-f231339815a1 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:33.068720433 +0000 UTC m=+143.085091769 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls") pod "openshift-state-metrics-84bd68b896-mcl6d" (UID: "f6766722-4419-4487-82cd-f231339815a1") : secret "openshift-state-metrics-tls" not found Jul 15 16:39:32.569173 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.569156 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f6766722-4419-4487-82cd-f231339815a1-metrics-client-ca\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.570708 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.570688 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.585376 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.585353 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmfr8\" (UniqueName: \"kubernetes.io/projected/f6766722-4419-4487-82cd-f231339815a1-kube-api-access-xmfr8\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:32.669540 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669515 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669540 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669542 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-sys\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669575 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-thgl2\" (UniqueName: \"kubernetes.io/projected/f1e906a7-ea5d-4393-9573-5461c28b2681-kube-api-access-thgl2\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669629 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-sys\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669692 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:32.669658 2566 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 15 16:39:32.669692 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669676 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-textfile\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669706 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:32.669714 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls podName:f1e906a7-ea5d-4393-9573-5461c28b2681 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:33.169700887 +0000 UTC m=+143.186072216 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls") pod "node-exporter-cxnqd" (UID: "f1e906a7-ea5d-4393-9573-5461c28b2681") : secret "node-exporter-tls" not found Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669745 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-root\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669794 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-wtmp\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669817 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-metrics-client-ca\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-root\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.669870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669842 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.670183 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.669911 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-wtmp\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.670183 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.670082 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-textfile\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.670299 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.670275 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-accelerators-collector-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.670371 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.670357 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f1e906a7-ea5d-4393-9573-5461c28b2681-metrics-client-ca\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.671921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.671904 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.683746 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.683727 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-thgl2\" (UniqueName: \"kubernetes.io/projected/f1e906a7-ea5d-4393-9573-5461c28b2681-kube-api-access-thgl2\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:32.730213 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.730178 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.832248 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.832170 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:32.936463 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:32.936430 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:33.040876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.040848 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:33.073215 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.073192 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:33.075538 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.075516 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f6766722-4419-4487-82cd-f231339815a1-openshift-state-metrics-tls\") pod \"openshift-state-metrics-84bd68b896-mcl6d\" (UID: \"f6766722-4419-4487-82cd-f231339815a1\") " pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:33.173956 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.173868 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:33.176080 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.176057 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/f1e906a7-ea5d-4393-9573-5461c28b2681-node-exporter-tls\") pod \"node-exporter-cxnqd\" (UID: \"f1e906a7-ea5d-4393-9573-5461c28b2681\") " pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:33.337680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.337642 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" Jul 15 16:39:33.374679 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.374616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:33.410694 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.410651 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cxnqd" Jul 15 16:39:33.422156 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:33.422123 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1e906a7_ea5d_4393_9573_5461c28b2681.slice/crio-88210f2aa2ed6bb247f5ddb01bdc1e43ed1d6bab3706dcda0639d5e0a42d8bcf WatchSource:0}: Error finding container 88210f2aa2ed6bb247f5ddb01bdc1e43ed1d6bab3706dcda0639d5e0a42d8bcf: Status 404 returned error can't find the container with id 88210f2aa2ed6bb247f5ddb01bdc1e43ed1d6bab3706dcda0639d5e0a42d8bcf Jul 15 16:39:33.468984 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.468943 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d"] Jul 15 16:39:33.470001 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.469979 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:33.472867 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:33.472835 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6766722_4419_4487_82cd_f231339815a1.slice/crio-fabd789c2193ad5142d277b93146640abc0cc9f1c019a68b6ae174d10da68425 WatchSource:0}: Error finding container fabd789c2193ad5142d277b93146640abc0cc9f1c019a68b6ae174d10da68425: Status 404 returned error can't find the container with id fabd789c2193ad5142d277b93146640abc0cc9f1c019a68b6ae174d10da68425 Jul 15 16:39:33.473936 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.473913 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 16:39:33.478240 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.478221 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.481337 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481299 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-7lq5d\"" Jul 15 16:39:33.481470 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481450 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\"" Jul 15 16:39:33.482046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481561 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\"" Jul 15 16:39:33.482046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481647 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\"" Jul 15 16:39:33.482046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481738 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\"" Jul 15 16:39:33.482046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481825 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\"" Jul 15 16:39:33.482046 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.481899 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\"" Jul 15 16:39:33.482375 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.482059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\"" Jul 15 16:39:33.482375 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.482059 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\"" Jul 15 16:39:33.483000 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.482879 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\"" Jul 15 16:39:33.499672 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.499651 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 16:39:33.575981 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.575955 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576022 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576052 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-out\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576113 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-web-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576203 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576252 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576280 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576350 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576308 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frtqf\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-kube-api-access-frtqf\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576643 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576388 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.576643 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.576451 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-volume\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.591563 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.591498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677451 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677518 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677555 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677603 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-out\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677627 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-web-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677659 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677686 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677716 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677767 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677792 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677819 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-frtqf\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-kube-api-access-frtqf\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677851 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.678429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.677875 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-volume\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.679144 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:33.678748 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle podName:aaabce8b-3db3-4eb0-acac-701e35db4d60 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:34.178727558 +0000 UTC m=+144.195098890 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "alertmanager-trusted-ca-bundle" (UniqueName: "kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle") pod "alertmanager-main-0" (UID: "aaabce8b-3db3-4eb0-acac-701e35db4d60") : configmap references non-existent config key: ca-bundle.crt Jul 15 16:39:33.679808 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:33.679428 2566 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found Jul 15 16:39:33.679808 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:33.679485 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls podName:aaabce8b-3db3-4eb0-acac-701e35db4d60 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:34.179469204 +0000 UTC m=+144.195840547 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "aaabce8b-3db3-4eb0-acac-701e35db4d60") : secret "alertmanager-main-tls" not found Jul 15 16:39:33.679808 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.679725 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.680357 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.680316 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.681099 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.681073 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-volume\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.682904 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.682857 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.683294 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.683277 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-web-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.683820 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.683800 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/aaabce8b-3db3-4eb0-acac-701e35db4d60-config-out\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.684738 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.684213 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-tls-assets\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.684738 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.684674 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.685793 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.685775 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.686103 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.686087 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.704428 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.704400 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-frtqf\" (UniqueName: \"kubernetes.io/projected/aaabce8b-3db3-4eb0-acac-701e35db4d60-kube-api-access-frtqf\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:33.789072 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:33.789032 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:34.005386 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.005348 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxnqd" event={"ID":"f1e906a7-ea5d-4393-9573-5461c28b2681","Type":"ContainerStarted","Data":"88210f2aa2ed6bb247f5ddb01bdc1e43ed1d6bab3706dcda0639d5e0a42d8bcf"} Jul 15 16:39:34.007825 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.007770 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" event={"ID":"f6766722-4419-4487-82cd-f231339815a1","Type":"ContainerStarted","Data":"721db90e09a29805b7171b910fdb9261f0dbfe16d2da3dde12734a9efb010377"} Jul 15 16:39:34.007971 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.007832 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" event={"ID":"f6766722-4419-4487-82cd-f231339815a1","Type":"ContainerStarted","Data":"4c17f529af6d6f81aa0e87cb45a532362bf480fa8ba422104f456dad7258de9c"} Jul 15 16:39:34.007971 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.007859 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" event={"ID":"f6766722-4419-4487-82cd-f231339815a1","Type":"ContainerStarted","Data":"fabd789c2193ad5142d277b93146640abc0cc9f1c019a68b6ae174d10da68425"} Jul 15 16:39:34.183642 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.183589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:34.184113 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.183662 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:34.184612 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.184584 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/aaabce8b-3db3-4eb0-acac-701e35db4d60-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:34.186724 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.186699 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/aaabce8b-3db3-4eb0-acac-701e35db4d60-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"aaabce8b-3db3-4eb0-acac-701e35db4d60\") " pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:34.391167 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.390878 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0" Jul 15 16:39:34.623451 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:34.623415 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"] Jul 15 16:39:34.829465 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:34.829433 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaaabce8b_3db3_4eb0_acac_701e35db4d60.slice/crio-612b3ae3cdf6023d3c5ccef03510c32a6dd9b296fc6d34cd28d3492401e09d91 WatchSource:0}: Error finding container 612b3ae3cdf6023d3c5ccef03510c32a6dd9b296fc6d34cd28d3492401e09d91: Status 404 returned error can't find the container with id 612b3ae3cdf6023d3c5ccef03510c32a6dd9b296fc6d34cd28d3492401e09d91 Jul 15 16:39:35.013745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.013704 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"612b3ae3cdf6023d3c5ccef03510c32a6dd9b296fc6d34cd28d3492401e09d91"} Jul 15 16:39:35.444638 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.444605 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:35.501378 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.501344 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-786b8476f7-ww2zf"] Jul 15 16:39:35.512178 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.512159 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.519141 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519112 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 15 16:39:35.519318 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519262 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 15 16:39:35.519578 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519555 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-co5b5suk7uknd\"" Jul 15 16:39:35.519728 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519578 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 15 16:39:35.519728 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519687 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:35.519845 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519779 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 15 16:39:35.520000 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.519984 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:35.520557 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.520291 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-44mt9\"" Jul 15 16:39:35.520557 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.520449 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 15 16:39:35.520701 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.520594 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786b8476f7-ww2zf"] Jul 15 16:39:35.521896 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.521871 2566 patch_prober.go:28] interesting pod/console-7d8b489f9c-j5sgv container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body= Jul 15 16:39:35.521994 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.521915 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d8b489f9c-j5sgv" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" probeResult="failure" output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" Jul 15 16:39:35.550399 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.550354 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:35.596143 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596114 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596346 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596303 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596438 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596386 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-metrics-client-ca\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596488 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596440 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596570 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596549 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596685 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596578 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-grpc-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596866 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596765 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.596866 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.596799 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7cfnd\" (UniqueName: \"kubernetes.io/projected/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-kube-api-access-7cfnd\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.649274 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.649234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:35.697727 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.697636 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.697727 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.697691 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7cfnd\" (UniqueName: \"kubernetes.io/projected/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-kube-api-access-7cfnd\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.697931 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.697728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698143 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698046 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698143 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698098 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-metrics-client-ca\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698301 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698140 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698301 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698211 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698301 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698252 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-grpc-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.698972 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.698911 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-metrics-client-ca\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.700778 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.700748 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.701567 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.701433 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.701659 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.701611 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.702073 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.702048 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.702171 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.702099 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-grpc-tls\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.702233 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.702179 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.710185 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.710159 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-7cfnd\" (UniqueName: \"kubernetes.io/projected/da5a4a1c-95b7-4a4b-a90b-136d4b12e252-kube-api-access-7cfnd\") pod \"thanos-querier-786b8476f7-ww2zf\" (UID: \"da5a4a1c-95b7-4a4b-a90b-136d4b12e252\") " pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:35.743633 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.743571 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:35.825284 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:35.825251 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:37.174488 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.174455 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:37.180996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.180971 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv"] Jul 15 16:39:37.184399 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.184377 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.187440 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.187414 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jul 15 16:39:37.187637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.187621 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-dockercfg-bgn69\"" Jul 15 16:39:37.196504 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.196451 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv"] Jul 15 16:39:37.212014 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.211981 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.212165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.212032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvt2l\" (UniqueName: \"kubernetes.io/projected/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-kube-api-access-hvt2l\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.303274 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.303217 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:37.313467 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.313442 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.313590 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.313486 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvt2l\" (UniqueName: \"kubernetes.io/projected/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-kube-api-access-hvt2l\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.313590 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:37.313573 2566 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jul 15 16:39:37.313691 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:37.313642 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert podName:7dbcd7d3-bb9a-4206-a788-d01e559e0ad8 nodeName:}" failed. No retries permitted until 2026-07-15 16:39:37.813627942 +0000 UTC m=+147.829999273 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert") pod "monitoring-plugin-5f68df7f58-bgshv" (UID: "7dbcd7d3-bb9a-4206-a788-d01e559e0ad8") : secret "monitoring-plugin-cert" not found Jul 15 16:39:37.325466 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.325434 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvt2l\" (UniqueName: \"kubernetes.io/projected/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-kube-api-access-hvt2l\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.404883 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.404825 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:37.458449 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.458208 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-786b8476f7-ww2zf"] Jul 15 16:39:37.490494 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:37.490464 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda5a4a1c_95b7_4a4b_a90b_136d4b12e252.slice/crio-32f2541bf31971a2f1ca18f90060b36b0eb956efcd827dd75f322be265769b5d WatchSource:0}: Error finding container 32f2541bf31971a2f1ca18f90060b36b0eb956efcd827dd75f322be265769b5d: Status 404 returned error can't find the container with id 32f2541bf31971a2f1ca18f90060b36b0eb956efcd827dd75f322be265769b5d Jul 15 16:39:37.504753 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.504728 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:37.818622 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.818597 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:37.820845 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:37.820828 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7dbcd7d3-bb9a-4206-a788-d01e559e0ad8-monitoring-plugin-cert\") pod \"monitoring-plugin-5f68df7f58-bgshv\" (UID: \"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8\") " pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:38.028603 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.028571 2566 generic.go:356] "Generic (PLEG): container finished" podID="f1e906a7-ea5d-4393-9573-5461c28b2681" containerID="8dd554010129dcdf36340b8a182ed6014815bcacc8df83e191130ca12527a823" exitCode=0 Jul 15 16:39:38.028745 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.028647 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxnqd" event={"ID":"f1e906a7-ea5d-4393-9573-5461c28b2681","Type":"ContainerDied","Data":"8dd554010129dcdf36340b8a182ed6014815bcacc8df83e191130ca12527a823"} Jul 15 16:39:38.030809 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.030784 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" event={"ID":"f6766722-4419-4487-82cd-f231339815a1","Type":"ContainerStarted","Data":"6b5074979846f94ba7ed25876db661848236041a661c9153982ce68f5d70e674"} Jul 15 16:39:38.031929 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.031906 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"32f2541bf31971a2f1ca18f90060b36b0eb956efcd827dd75f322be265769b5d"} Jul 15 16:39:38.033665 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.033645 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lczwb" event={"ID":"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00","Type":"ContainerStarted","Data":"4617dd3067e96c365cf711eaebdd0f065fea6dd6f834ceef1cc586d18e13b56a"} Jul 15 16:39:38.034965 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.034944 2566 generic.go:356] "Generic (PLEG): container finished" podID="aaabce8b-3db3-4eb0-acac-701e35db4d60" containerID="a2aece2c18a741e76febd16b422756ce6ad483c42dad9b32789762dac7e552b9" exitCode=0 Jul 15 16:39:38.035041 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.034972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerDied","Data":"a2aece2c18a741e76febd16b422756ce6ad483c42dad9b32789762dac7e552b9"} Jul 15 16:39:38.075357 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.075294 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-84bd68b896-mcl6d" podStartSLOduration=2.425114888 podStartE2EDuration="6.075278814s" totalImagesPullingTime="3.650163926s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:38.07473486 +0000 UTC m=+148.091106214" watchObservedRunningTime="2026-07-15 16:39:38.075278814 +0000 UTC m=+148.091650164" Jul 15 16:39:38.094580 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.094559 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:38.233758 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.233691 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv"] Jul 15 16:39:38.235811 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:38.235789 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dbcd7d3_bb9a_4206_a788_d01e559e0ad8.slice/crio-878eb4de37dfb972040a8cca5a07c38999339a111bbcac51f4065d69da363f37 WatchSource:0}: Error finding container 878eb4de37dfb972040a8cca5a07c38999339a111bbcac51f4065d69da363f37: Status 404 returned error can't find the container with id 878eb4de37dfb972040a8cca5a07c38999339a111bbcac51f4065d69da363f37 Jul 15 16:39:38.765347 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:38.765295 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:39.041982 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.041901 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxnqd" event={"ID":"f1e906a7-ea5d-4393-9573-5461c28b2681","Type":"ContainerStarted","Data":"764578479efdd7080a77b3142f78da64103eb59badb2920cef52669a525b5fc0"} Jul 15 16:39:39.042122 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.042008 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cxnqd" event={"ID":"f1e906a7-ea5d-4393-9573-5461c28b2681","Type":"ContainerStarted","Data":"da979635045da9e219b0f120d07eccba73edb121193479cbf832305ca09d9081"} Jul 15 16:39:39.044224 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.044164 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" event={"ID":"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8","Type":"ContainerStarted","Data":"878eb4de37dfb972040a8cca5a07c38999339a111bbcac51f4065d69da363f37"} Jul 15 16:39:39.065800 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.065743 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cxnqd" podStartSLOduration=3.382224472 podStartE2EDuration="7.065716944s" totalImagesPullingTime="3.610818179s" totalInitContainerRuntime="72.674293ms" isStatefulPod=true podCreationTimestamp="2026-07-15 16:39:32 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:39.064389085 +0000 UTC m=+149.080760437" watchObservedRunningTime="2026-07-15 16:39:39.065716944 +0000 UTC m=+149.082088296" Jul 15 16:39:39.195895 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.195859 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:39.196067 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:39.195912 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:39:40.948292 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:40.948198 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:40.948292 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:40.948265 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:39:41.059384 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.059316 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"283371ca9fd229fe5c960b4f48fa71605040c543b35704f488e9eddcea77502c"} Jul 15 16:39:41.059561 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.059449 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"4aed65cd365ac75e8202ca87c52400b1ae8eb8888b01d33b7cb597ff545ad16e"} Jul 15 16:39:41.059561 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.059491 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"55a6bbce532bd8c218dda70b550c0ca53bb6d7ae7b324bee0079558fd8f86ddd"} Jul 15 16:39:41.061022 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.060997 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" event={"ID":"7dbcd7d3-bb9a-4206-a788-d01e559e0ad8","Type":"ContainerStarted","Data":"6430f3cfba28723e8e1b0e3c3239bed16d9166b8abb8d5596bfbd077b6cbb288"} Jul 15 16:39:41.061231 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.061216 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:41.066620 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.066597 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" Jul 15 16:39:41.081737 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:41.081698 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5f68df7f58-bgshv" podStartSLOduration=1.6517280680000002 podStartE2EDuration="4.081686382s" totalImagesPullingTime="2.429958314s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:37 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:41.080341157 +0000 UTC m=+151.096712509" watchObservedRunningTime="2026-07-15 16:39:41.081686382 +0000 UTC m=+151.098057711" Jul 15 16:39:44.074081 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.074042 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lczwb" event={"ID":"fafc9049-86af-4af5-b9ed-cd5a5ad2fc00","Type":"ContainerStarted","Data":"a7073f0fafe11e52e18b1a7a37bd3f70a159feda461f53dd092081d48168cf46"} Jul 15 16:39:44.076713 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.076688 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"e0545baaf93a991a0d4c0c5b1bc68da451704a7526d1fb2f5e8f215448e477f1"} Jul 15 16:39:44.076846 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.076756 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"6a09cdee2f752eb294af935071faa240916e50a0f8489c57ce8fcaa63501df49"} Jul 15 16:39:44.076846 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.076797 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" event={"ID":"da5a4a1c-95b7-4a4b-a90b-136d4b12e252","Type":"ContainerStarted","Data":"4e2c5a503fdd4ef969390f062ef6eaf2bd8f3d92afc5d54dcdc3755465c264ba"} Jul 15 16:39:44.076846 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.076837 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:44.079813 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.079782 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"b23d65057515ddb39b960c290088a8dd8ed6e026ddde63ea2b443f03d0b93d60"} Jul 15 16:39:44.079939 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.079848 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"b55c6777e1eaffee7c9551124d9584e3b8d6f17ab0ac63f02ff89d9f9c17b3fd"} Jul 15 16:39:44.079939 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.079887 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"75d35dc05a09f2b286cd92e4fa2efc05b66bd4ec8c8564e1a4d3544579aa5b6f"} Jul 15 16:39:44.079939 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.079924 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"ee044c2eee5601052e218b8001b8c22794b4f3890a2d87ca59cde1eac7744a33"} Jul 15 16:39:44.080082 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.079963 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"c76c48e7734ae7e62cfd14ebdd8e8e024eb4df0892d42865da4d719e284ba204"} Jul 15 16:39:44.080082 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.080004 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"aaabce8b-3db3-4eb0-acac-701e35db4d60","Type":"ContainerStarted","Data":"0030ca3e4ca368e33ddda119d5a632433ce8e444d1536229317f97dd37f2c539"} Jul 15 16:39:44.094895 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.094852 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lczwb" podStartSLOduration=34.079871679 podStartE2EDuration="52.09483798s" totalImagesPullingTime="18.014966301s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 16:38:52 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:44.093864151 +0000 UTC m=+154.110235503" watchObservedRunningTime="2026-07-15 16:39:44.09483798 +0000 UTC m=+154.111209330" Jul 15 16:39:44.121820 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.121781 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=3.444815438 podStartE2EDuration="11.121767169s" totalImagesPullingTime="7.652637177s" totalInitContainerRuntime="24.314554ms" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:33 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:44.120093335 +0000 UTC m=+154.136464687" watchObservedRunningTime="2026-07-15 16:39:44.121767169 +0000 UTC m=+154.138138564" Jul 15 16:39:44.143892 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:44.143847 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" podStartSLOduration=3.635485561 podStartE2EDuration="9.143832388s" totalImagesPullingTime="5.508346827s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:39:35 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:44.142687955 +0000 UTC m=+154.159059307" watchObservedRunningTime="2026-07-15 16:39:44.143832388 +0000 UTC m=+154.160203739" Jul 15 16:39:45.090006 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:45.089979 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-786b8476f7-ww2zf" Jul 15 16:39:45.520727 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:45.520694 2566 patch_prober.go:28] interesting pod/console-7d8b489f9c-j5sgv container/console namespace/openshift-console: Startup probe status=failure output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" start-of-body= Jul 15 16:39:45.520877 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:45.520759 2566 prober.go:120] "Probe failed" probeType="Startup" pod="openshift-console/console-7d8b489f9c-j5sgv" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" probeResult="failure" output="Get \"https://10.134.0.16:8443/health\": dial tcp 10.134.0.16:8443: connect: connection refused" Jul 15 16:39:46.977264 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:46.977229 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:47.083831 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:47.083797 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:49.195584 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:49.195548 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:49.195955 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:49.195598 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:39:50.945937 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:50.945846 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:50.945937 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:50.945900 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:39:50.987439 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:39:50.987401 2566 pod_workers.go:1324] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-ml26l" podUID="280bfa80-5182-48de-8e1e-a4dae02247fd" Jul 15 16:39:51.105511 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:51.105485 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ml26l" Jul 15 16:39:51.193782 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:51.193749 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:51.258445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:51.258411 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:55.524378 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.524348 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:55.528071 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.528050 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:39:55.750232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.750199 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:55.815736 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.815674 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:55.873461 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.873430 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:39:55.875601 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.875586 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/280bfa80-5182-48de-8e1e-a4dae02247fd-metrics-tls\") pod \"dns-default-ml26l\" (UID: \"280bfa80-5182-48de-8e1e-a4dae02247fd\") " pod="openshift-dns/dns-default-ml26l" Jul 15 16:39:55.909441 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.909419 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hj6qd\"" Jul 15 16:39:55.916851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:55.916834 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-ml26l" Jul 15 16:39:56.044056 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:56.044037 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-ml26l"] Jul 15 16:39:56.046071 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:39:56.046043 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod280bfa80_5182_48de_8e1e_a4dae02247fd.slice/crio-7fdde8615a4aea3adedcf779787930858cab26bf83f38102f361aae3f526143d WatchSource:0}: Error finding container 7fdde8615a4aea3adedcf779787930858cab26bf83f38102f361aae3f526143d: Status 404 returned error can't find the container with id 7fdde8615a4aea3adedcf779787930858cab26bf83f38102f361aae3f526143d Jul 15 16:39:56.126012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:56.125942 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml26l" event={"ID":"280bfa80-5182-48de-8e1e-a4dae02247fd","Type":"ContainerStarted","Data":"7fdde8615a4aea3adedcf779787930858cab26bf83f38102f361aae3f526143d"} Jul 15 16:39:56.367267 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:56.367236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:56.441257 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:56.441186 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:58.655976 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:58.655904 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:39:58.791524 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:58.791493 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:58.892084 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:58.892055 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:58.957947 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:58.957877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:39:59.145763 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.145727 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml26l" event={"ID":"280bfa80-5182-48de-8e1e-a4dae02247fd","Type":"ContainerStarted","Data":"12315ea9ad7bee52149525699488f1b5b721ecf7ae535b45f2d68cc2d02ce3d8"} Jul 15 16:39:59.145918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.145810 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-ml26l" event={"ID":"280bfa80-5182-48de-8e1e-a4dae02247fd","Type":"ContainerStarted","Data":"6689c0182ac219342ddc340a976652de8ab424a762c909a1aad2be121ff442cd"} Jul 15 16:39:59.145959 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.145929 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-ml26l" Jul 15 16:39:59.166637 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.166590 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-ml26l" podStartSLOduration=129.959002146 podStartE2EDuration="2m12.166575866s" totalImagesPullingTime="2.20757372s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:37:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:39:59.165924904 +0000 UTC m=+169.182296265" watchObservedRunningTime="2026-07-15 16:39:59.166575866 +0000 UTC m=+169.182947217" Jul 15 16:39:59.195540 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.195508 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:59.195664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.195561 2566 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:39:59.195664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.195599 2566 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:39:59.196089 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.196054 2566 kuberuntime_manager.go:1192] "Message for Container of pod" containerName="registry" containerStatusID={"Type":"cri-o","ID":"3eb5637d7755a0184cf4c7fab99c446ba2a183ca73f2fa65f369315732ea6972"} pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" containerMessage="Container registry failed liveness probe, will be restarted" Jul 15 16:39:59.199607 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.199578 2566 patch_prober.go:28] interesting pod/image-registry-6955c964f4-5tqp7 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 15 16:39:59.199718 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:39:59.199617 2566 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 15 16:40:00.020768 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:00.020739 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:00.089411 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:00.089383 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:09.152246 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:09.152171 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-ml26l" Jul 15 16:40:09.200565 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:09.200535 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:40:09.393496 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:09.393467 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:09.498545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:09.498508 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:09.595534 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:09.595507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:11.319794 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:11.319761 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:11.422845 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:11.422814 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:13.795742 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:13.795703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:13.896538 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:13.896509 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:14.531292 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:14.531249 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:14.632035 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:14.632005 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:21.294412 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:21.294378 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:21.403394 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:21.403362 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:23.675917 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.675883 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-7d8b489f9c-j5sgv" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" containerID="cri-o://a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e" gracePeriod=15 Jul 15 16:40:23.721365 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.721344 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-698f89cbfd-mss4k_bf3eae5b-98c4-4248-a052-44f3518fff74/router/0.log" Jul 15 16:40:23.736092 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.736073 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m7z8z_d031ba50-1ba1-46c5-9ab7-28268c46b659/serve-healthcheck-canary/0.log" Jul 15 16:40:23.778137 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.778109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:23.879319 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.879293 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:23.916405 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.916386 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:40:23.988464 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.988441 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:23.988610 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.988474 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkf57\" (UniqueName: \"kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:23.988610 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.988496 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:23.988679 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.988608 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:23.988831 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.988808 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca" (OuterVolumeSpecName: "service-ca") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:40:23.990748 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.990723 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57" (OuterVolumeSpecName: "kube-api-access-gkf57") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "kube-api-access-gkf57". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:40:23.990857 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.990730 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:40:23.990857 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:23.990760 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:40:24.089840 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.089817 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:24.089987 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.089858 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:24.089987 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.089877 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config\") pod \"fd2ba17b-9c4e-4338-95d8-676068b300f8\" (UID: \"fd2ba17b-9c4e-4338-95d8-676068b300f8\") " Jul 15 16:40:24.090102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.089993 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-service-ca\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.090102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090003 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkf57\" (UniqueName: \"kubernetes.io/projected/fd2ba17b-9c4e-4338-95d8-676068b300f8-kube-api-access-gkf57\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.090102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090012 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-oauth-config\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.090102 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090022 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-serving-cert\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.090288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090234 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:40:24.090288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090250 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config" (OuterVolumeSpecName: "console-config") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:40:24.090288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.090257 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "fd2ba17b-9c4e-4338-95d8-676068b300f8" (UID: "fd2ba17b-9c4e-4338-95d8-676068b300f8"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:40:24.191365 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.191321 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-trusted-ca-bundle\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.191365 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.191366 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-console-config\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.191538 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.191376 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/fd2ba17b-9c4e-4338-95d8-676068b300f8-oauth-serving-cert\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:40:24.215657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.215620 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" podUID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerName="registry" containerID="cri-o://3eb5637d7755a0184cf4c7fab99c446ba2a183ca73f2fa65f369315732ea6972" gracePeriod=30 Jul 15 16:40:24.248288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.248229 2566 generic.go:356] "Generic (PLEG): container finished" podID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerID="a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e" exitCode=0 Jul 15 16:40:24.248288 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.248275 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7d8b489f9c-j5sgv" Jul 15 16:40:24.248426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.248315 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8b489f9c-j5sgv" event={"ID":"fd2ba17b-9c4e-4338-95d8-676068b300f8","Type":"ContainerDied","Data":"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e"} Jul 15 16:40:24.248426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.248389 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7d8b489f9c-j5sgv" event={"ID":"fd2ba17b-9c4e-4338-95d8-676068b300f8","Type":"ContainerDied","Data":"beb4cb031c79daad89fd822f15ba1e7303f134254c6dbae97e2b2463a6bc5b57"} Jul 15 16:40:24.248426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.248415 2566 scope.go:117] "RemoveContainer" containerID="a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e" Jul 15 16:40:24.257317 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.257301 2566 scope.go:117] "RemoveContainer" containerID="a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e" Jul 15 16:40:24.257620 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:40:24.257601 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e\": container with ID starting with a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e not found: ID does not exist" containerID="a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e" Jul 15 16:40:24.257667 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.257628 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e"} err="failed to get container status \"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e\": rpc error: code = NotFound desc = could not find container \"a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e\": container with ID starting with a01375b3fe4d8b6f51aac9f16f3c5c01862fadb6d233cd1cb513465233dbf67e not found: ID does not exist" Jul 15 16:40:24.281568 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.281543 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:40:24.285250 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.285230 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7d8b489f9c-j5sgv"] Jul 15 16:40:24.522793 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:24.522719 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" path="/var/lib/kubelet/pods/fd2ba17b-9c4e-4338-95d8-676068b300f8/volumes" Jul 15 16:40:25.043677 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:25.043648 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:25.223243 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:25.223211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:25.409962 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:25.409939 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:26.258173 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.258143 2566 generic.go:356] "Generic (PLEG): container finished" podID="91e76f29-ffd4-4dd8-be9d-bdafa83e7dec" containerID="3eb5637d7755a0184cf4c7fab99c446ba2a183ca73f2fa65f369315732ea6972" exitCode=0 Jul 15 16:40:26.258571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.258185 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" event={"ID":"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec","Type":"ContainerDied","Data":"3eb5637d7755a0184cf4c7fab99c446ba2a183ca73f2fa65f369315732ea6972"} Jul 15 16:40:26.258571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.258217 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" event={"ID":"91e76f29-ffd4-4dd8-be9d-bdafa83e7dec","Type":"ContainerStarted","Data":"bba6ca8e65cea3f58ac51a4c810599fd0e76af8945b8c2c51443ff878c86428a"} Jul 15 16:40:26.258571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.258256 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:40:26.571363 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.571265 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:26.673899 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:26.673864 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:30.341618 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:30.341582 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:30.428247 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:30.428211 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:44.643722 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:44.643684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:44.797141 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:44.797105 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:47.267772 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:47.267745 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-6955c964f4-5tqp7" Jul 15 16:40:47.545994 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:47.545908 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:48.172301 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:48.172263 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:52.006214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:52.006175 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:52.095785 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:52.095750 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:56.874927 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.874892 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-86656545ff-lwz8d"] Jul 15 16:40:56.875379 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.875229 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" Jul 15 16:40:56.875379 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.875241 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" Jul 15 16:40:56.875379 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.875297 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="fd2ba17b-9c4e-4338-95d8-676068b300f8" containerName="console" Jul 15 16:40:56.879307 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.879288 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.881859 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.881835 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 15 16:40:56.881965 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.881900 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 15 16:40:56.882021 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.881961 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 15 16:40:56.882021 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.881967 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 15 16:40:56.882021 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.882000 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 15 16:40:56.882337 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.882308 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-kdtvb\"" Jul 15 16:40:56.889414 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.889396 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 15 16:40:56.893824 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.893806 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86656545ff-lwz8d"] Jul 15 16:40:56.957518 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957485 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957518 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957521 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-federate-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957539 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957564 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957602 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-serving-certs-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957636 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s2c8p\" (UniqueName: \"kubernetes.io/projected/d46b1191-0831-4629-9444-b7c326519fb9-kube-api-access-s2c8p\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957690 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:56.957917 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:56.957714 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-metrics-client-ca\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059366 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059314 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059519 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059394 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-serving-certs-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059519 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059439 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-s2c8p\" (UniqueName: \"kubernetes.io/projected/d46b1191-0831-4629-9444-b7c326519fb9-kube-api-access-s2c8p\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059639 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059559 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059639 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059589 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-metrics-client-ca\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059743 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059637 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059743 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059661 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-federate-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.059743 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.059678 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.060198 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.060174 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-serving-certs-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.060387 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.060230 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-trusted-ca-bundle\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.060387 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.060292 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/d46b1191-0831-4629-9444-b7c326519fb9-metrics-client-ca\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.062248 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.062227 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.062248 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.062239 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-federate-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.062554 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.062532 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.062592 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.062551 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/d46b1191-0831-4629-9444-b7c326519fb9-telemeter-client-tls\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.068174 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.068154 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-s2c8p\" (UniqueName: \"kubernetes.io/projected/d46b1191-0831-4629-9444-b7c326519fb9-kube-api-access-s2c8p\") pod \"telemeter-client-86656545ff-lwz8d\" (UID: \"d46b1191-0831-4629-9444-b7c326519fb9\") " pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.188686 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.188597 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" Jul 15 16:40:57.324734 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.324697 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-86656545ff-lwz8d"] Jul 15 16:40:57.328371 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:40:57.328308 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd46b1191_0831_4629_9444_b7c326519fb9.slice/crio-699d565158a5de4187149a279c86906aeccaff9de12b2273bb79f481cd9873b3 WatchSource:0}: Error finding container 699d565158a5de4187149a279c86906aeccaff9de12b2273bb79f481cd9873b3: Status 404 returned error can't find the container with id 699d565158a5de4187149a279c86906aeccaff9de12b2273bb79f481cd9873b3 Jul 15 16:40:57.380395 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:57.380356 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" event={"ID":"d46b1191-0831-4629-9444-b7c326519fb9","Type":"ContainerStarted","Data":"699d565158a5de4187149a279c86906aeccaff9de12b2273bb79f481cd9873b3"} Jul 15 16:40:59.518098 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:59.518057 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:40:59.686565 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:40:59.686531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:00.394353 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:00.394300 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" event={"ID":"d46b1191-0831-4629-9444-b7c326519fb9","Type":"ContainerStarted","Data":"e9474a5ce4fe7f9345c8efe48adad6b966582aaebf8ab2f636d8ada210bb81ac"} Jul 15 16:41:00.394545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:00.394376 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" event={"ID":"d46b1191-0831-4629-9444-b7c326519fb9","Type":"ContainerStarted","Data":"c65bca75b0fb3f89becea391421f8555d88ed4ce5245cba657ed8c7191cd66a1"} Jul 15 16:41:00.394545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:00.394395 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" event={"ID":"d46b1191-0831-4629-9444-b7c326519fb9","Type":"ContainerStarted","Data":"50b56d827171ce7a18ca8c49a29dd990b180293b32dc6540eab0c688ab2b2d6f"} Jul 15 16:41:00.418661 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:00.418615 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-86656545ff-lwz8d" podStartSLOduration=1.8548681120000001 podStartE2EDuration="4.418599738s" totalImagesPullingTime="2.563731626s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:40:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:41:00.4174268 +0000 UTC m=+230.433798151" watchObservedRunningTime="2026-07-15 16:41:00.418599738 +0000 UTC m=+230.434971089" Jul 15 16:41:01.117493 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.117421 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:01.119888 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.119871 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.123799 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.123774 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 16:41:01.123956 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.123861 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 16:41:01.123956 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.123887 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 16:41:01.123956 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.123925 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 16:41:01.124153 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.123973 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 16:41:01.124153 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.124001 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mtjsr\"" Jul 15 16:41:01.124231 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.124179 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 16:41:01.124282 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.124268 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 16:41:01.128891 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.128873 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 16:41:01.134406 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.134386 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:01.173542 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.173507 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:01.194644 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194616 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194744 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194659 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194744 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194686 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194859 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194760 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194859 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194786 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-srzm8\" (UniqueName: \"kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194859 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194830 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.194962 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.194910 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.246939 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.246915 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:01.296073 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296051 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296087 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296244 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296219 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296287 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296275 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296423 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296307 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-srzm8\" (UniqueName: \"kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296472 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296461 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.296884 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.296861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.297092 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.297073 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.297166 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.297112 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.297237 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.297219 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.298593 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.298573 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.298870 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.298850 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.305675 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.305653 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-srzm8\" (UniqueName: \"kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8\") pod \"console-86b97b598-ht5jb\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.429876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.429817 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:01.554852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:01.554817 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:01.557648 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:41:01.557626 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod09942735_6fa9_4d8c_9cfc_55d0a49c5d0f.slice/crio-365e889368accd26c899679c005252a3ba66ce1ed2c488fa88acc630b7442c8b WatchSource:0}: Error finding container 365e889368accd26c899679c005252a3ba66ce1ed2c488fa88acc630b7442c8b: Status 404 returned error can't find the container with id 365e889368accd26c899679c005252a3ba66ce1ed2c488fa88acc630b7442c8b Jul 15 16:41:02.403649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:02.403621 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b97b598-ht5jb" event={"ID":"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f","Type":"ContainerStarted","Data":"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67"} Jul 15 16:41:02.404065 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:02.403667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b97b598-ht5jb" event={"ID":"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f","Type":"ContainerStarted","Data":"365e889368accd26c899679c005252a3ba66ce1ed2c488fa88acc630b7442c8b"} Jul 15 16:41:02.423272 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:02.423222 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-86b97b598-ht5jb" podStartSLOduration=1.4232065440000001 podStartE2EDuration="1.423206544s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:41:01 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:41:02.422155997 +0000 UTC m=+232.438527340" watchObservedRunningTime="2026-07-15 16:41:02.423206544 +0000 UTC m=+232.439577894" Jul 15 16:41:05.571047 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:05.571016 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:05.636655 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:05.636619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:07.019092 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:07.019056 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:07.120219 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:07.120181 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:10.857264 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:10.857234 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:11.430992 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:11.430953 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:12.654019 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:12.653988 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:12.714620 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:12.714582 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:22.219177 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:22.219148 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:22.320557 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:22.320530 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:29.530369 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:29.530339 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:32.023534 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:32.023498 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:32.082953 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:32.082884 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:35.876830 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:35.876795 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-console/console-86b97b598-ht5jb" podUID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" containerName="console" containerID="cri-o://fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67" gracePeriod=15 Jul 15 16:41:36.117762 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.117739 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:36.164214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164133 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164165 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164181 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-srzm8\" (UniqueName: \"kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164499 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164228 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164499 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164272 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164499 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164345 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164499 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164378 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config\") pod \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\" (UID: \"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f\") " Jul 15 16:41:36.164710 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164600 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:41:36.164710 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164607 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config" (OuterVolumeSpecName: "console-config") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:41:36.165042 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.164984 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:41:36.165173 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.165149 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca" (OuterVolumeSpecName: "service-ca") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:41:36.166548 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.166524 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:41:36.166942 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.166922 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:41:36.166987 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.166950 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8" (OuterVolumeSpecName: "kube-api-access-srzm8") pod "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" (UID: "09942735-6fa9-4d8c-9cfc-55d0a49c5d0f"). InnerVolumeSpecName "kube-api-access-srzm8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:41:36.265229 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265203 2566 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-serving-cert\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265229 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265225 2566 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-oauth-config\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265229 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265234 2566 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-trusted-ca-bundle\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265433 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265244 2566 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-console-config\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265433 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265252 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-srzm8\" (UniqueName: \"kubernetes.io/projected/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-kube-api-access-srzm8\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265433 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265262 2566 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-oauth-serving-cert\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.265433 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.265270 2566 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f-service-ca\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:41:36.535115 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.535084 2566 generic.go:356] "Generic (PLEG): container finished" podID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" containerID="fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67" exitCode=0 Jul 15 16:41:36.535239 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.535138 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-86b97b598-ht5jb" Jul 15 16:41:36.535239 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.535152 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b97b598-ht5jb" event={"ID":"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f","Type":"ContainerDied","Data":"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67"} Jul 15 16:41:36.535239 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.535195 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-86b97b598-ht5jb" event={"ID":"09942735-6fa9-4d8c-9cfc-55d0a49c5d0f","Type":"ContainerDied","Data":"365e889368accd26c899679c005252a3ba66ce1ed2c488fa88acc630b7442c8b"} Jul 15 16:41:36.535239 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.535222 2566 scope.go:117] "RemoveContainer" containerID="fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67" Jul 15 16:41:36.543368 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.543351 2566 scope.go:117] "RemoveContainer" containerID="fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67" Jul 15 16:41:36.543613 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:41:36.543593 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67\": container with ID starting with fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67 not found: ID does not exist" containerID="fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67" Jul 15 16:41:36.543691 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.543618 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67"} err="failed to get container status \"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67\": rpc error: code = NotFound desc = could not find container \"fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67\": container with ID starting with fefcf05f2f6e3281da8dfd4b0747a6c2d33482d95143e22163d6ca4a520d6d67 not found: ID does not exist" Jul 15 16:41:36.553999 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.553976 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:36.556410 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:36.556390 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-86b97b598-ht5jb"] Jul 15 16:41:38.521957 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:38.521924 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" path="/var/lib/kubelet/pods/09942735-6fa9-4d8c-9cfc-55d0a49c5d0f/volumes" Jul 15 16:41:53.961076 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:53.961014 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:41:54.191164 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:41:54.191135 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:10.350633 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:10.350601 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:42:10.353897 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:10.353870 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:42:10.356811 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:10.356788 2566 kubelet.go:1648] "Image garbage collection succeeded" Jul 15 16:42:17.274052 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:17.274003 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:17.400266 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:17.400229 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:44.531407 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:44.531372 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:44.860657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:44.860563 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:48.541711 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:48.541669 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:48.641604 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:48.641570 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:49.369267 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:49.369234 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:49.587341 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:49.587289 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:49.853357 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:49.853306 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:50.080123 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:50.080086 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:51.717532 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:51.717504 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:52.415338 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:52.415292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:55.214816 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:55.214788 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:42:55.338221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:42:55.338193 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:00.709716 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:00.709684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:01.070996 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:01.070969 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:01.140803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:01.140773 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:02.483903 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.483829 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5gr5m"] Jul 15 16:43:02.484241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.484160 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" containerName="console" Jul 15 16:43:02.484241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.484171 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" containerName="console" Jul 15 16:43:02.484241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.484226 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="09942735-6fa9-4d8c-9cfc-55d0a49c5d0f" containerName="console" Jul 15 16:43:02.491265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.491247 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.496243 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.496222 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-dockercfg-78qp4\"" Jul 15 16:43:02.496554 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.496536 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\"" Jul 15 16:43:02.497542 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.497384 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\"" Jul 15 16:43:02.509961 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.509941 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5gr5m"] Jul 15 16:43:02.532482 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.532460 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr7lk\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-kube-api-access-wr7lk\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.532589 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.532520 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-bound-sa-token\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.633722 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.633699 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wr7lk\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-kube-api-access-wr7lk\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.633871 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.633753 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-bound-sa-token\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.646688 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.646659 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-bound-sa-token\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.646837 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.646820 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr7lk\" (UniqueName: \"kubernetes.io/projected/7a2f1140-aebc-48c7-b62a-78a49381d52c-kube-api-access-wr7lk\") pod \"cert-manager-55b4d954d5-5gr5m\" (UID: \"7a2f1140-aebc-48c7-b62a-78a49381d52c\") " pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.811481 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.811412 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-55b4d954d5-5gr5m" Jul 15 16:43:02.944649 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.944621 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-55b4d954d5-5gr5m"] Jul 15 16:43:02.946291 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:02.946270 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a2f1140_aebc_48c7_b62a_78a49381d52c.slice/crio-4782ba7a177df7dbcbaf5edf6130d1f17df72fa02845d4cd63dc90b75b240c39 WatchSource:0}: Error finding container 4782ba7a177df7dbcbaf5edf6130d1f17df72fa02845d4cd63dc90b75b240c39: Status 404 returned error can't find the container with id 4782ba7a177df7dbcbaf5edf6130d1f17df72fa02845d4cd63dc90b75b240c39 Jul 15 16:43:02.948222 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:02.948202 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:43:03.865825 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:03.865790 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5gr5m" event={"ID":"7a2f1140-aebc-48c7-b62a-78a49381d52c","Type":"ContainerStarted","Data":"4782ba7a177df7dbcbaf5edf6130d1f17df72fa02845d4cd63dc90b75b240c39"} Jul 15 16:43:04.705070 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:04.704965 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:04.802009 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:04.801964 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:06.880467 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:06.880432 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-55b4d954d5-5gr5m" event={"ID":"7a2f1140-aebc-48c7-b62a-78a49381d52c","Type":"ContainerStarted","Data":"5712f2f7267da5115931e5a54eb98585f9a643202b3a9c2342090b70e5bace82"} Jul 15 16:43:06.911347 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:06.911210 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="cert-manager/cert-manager-55b4d954d5-5gr5m" podStartSLOduration=1.431804208 podStartE2EDuration="4.91119046s" totalImagesPullingTime="3.479386252s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:02 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:06.911012477 +0000 UTC m=+356.927383827" watchObservedRunningTime="2026-07-15 16:43:06.91119046 +0000 UTC m=+356.927561811" Jul 15 16:43:07.714078 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:07.714047 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:10.590913 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:10.590867 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:10.694290 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:10.694256 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:21.936252 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:21.936204 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:22.124010 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:22.123977 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:23.002142 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.002109 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s"] Jul 15 16:43:23.005576 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.005557 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.010089 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.010070 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jul 15 16:43:23.010195 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.010181 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-wjh8v\"" Jul 15 16:43:23.010395 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.010383 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jul 15 16:43:23.010442 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.010419 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jul 15 16:43:23.027941 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.027918 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jul 15 16:43:23.097958 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.097925 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-manifests\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.098124 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.097970 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-tmp\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.098124 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.098032 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vfsqx\" (UniqueName: \"kubernetes.io/projected/e2477bfc-22c7-4f48-85f0-3b13443b5598-kube-api-access-vfsqx\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.098124 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.098073 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.098124 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.098105 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.198664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.198632 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.198828 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.198671 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.198828 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.198728 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-manifests\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.198828 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.198754 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-tmp\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.198828 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.198791 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vfsqx\" (UniqueName: \"kubernetes.io/projected/e2477bfc-22c7-4f48-85f0-3b13443b5598-kube-api-access-vfsqx\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.199112 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.199093 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-manifests\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.201012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.200971 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/e2477bfc-22c7-4f48-85f0-3b13443b5598-tmp\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.201162 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.201141 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-apiservice-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.201214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.201146 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/e2477bfc-22c7-4f48-85f0-3b13443b5598-webhook-cert\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.235134 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.235111 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-vfsqx\" (UniqueName: \"kubernetes.io/projected/e2477bfc-22c7-4f48-85f0-3b13443b5598-kube-api-access-vfsqx\") pod \"opendatahub-operator-controller-manager-67f48c5d8-4nl2s\" (UID: \"e2477bfc-22c7-4f48-85f0-3b13443b5598\") " pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.245807 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.245777 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s"] Jul 15 16:43:23.315415 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.315347 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:23.464250 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.464219 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s"] Jul 15 16:43:23.469780 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:23.469754 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2477bfc_22c7_4f48_85f0_3b13443b5598.slice/crio-db45deca291b67a03fc044451d5bee4262019075d7dd4523490d3029534c6a24 WatchSource:0}: Error finding container db45deca291b67a03fc044451d5bee4262019075d7dd4523490d3029534c6a24: Status 404 returned error can't find the container with id db45deca291b67a03fc044451d5bee4262019075d7dd4523490d3029534c6a24 Jul 15 16:43:23.947680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:23.947643 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" event={"ID":"e2477bfc-22c7-4f48-85f0-3b13443b5598","Type":"ContainerStarted","Data":"db45deca291b67a03fc044451d5bee4262019075d7dd4523490d3029534c6a24"} Jul 15 16:43:24.453719 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:24.453687 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:24.555387 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:24.555315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:25.361999 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:25.361961 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:26.972958 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:26.972920 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" event={"ID":"e2477bfc-22c7-4f48-85f0-3b13443b5598","Type":"ContainerStarted","Data":"65dfbf9fcb96a9a4bf2d51784e48df70d2598a27d0e2e5e3b704f0cdb5749901"} Jul 15 16:43:27.978206 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:27.978167 2566 generic.go:356] "Generic (PLEG): container finished" podID="e2477bfc-22c7-4f48-85f0-3b13443b5598" containerID="65dfbf9fcb96a9a4bf2d51784e48df70d2598a27d0e2e5e3b704f0cdb5749901" exitCode=0 Jul 15 16:43:27.978636 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:27.978237 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" event={"ID":"e2477bfc-22c7-4f48-85f0-3b13443b5598","Type":"ContainerDied","Data":"65dfbf9fcb96a9a4bf2d51784e48df70d2598a27d0e2e5e3b704f0cdb5749901"} Jul 15 16:43:28.985024 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:28.984980 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" event={"ID":"e2477bfc-22c7-4f48-85f0-3b13443b5598","Type":"ContainerStarted","Data":"99d80bdda58eb6c9da65bc4317caa4098a9bcd54fceb32d81b090ab695f88a00"} Jul 15 16:43:28.985526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:28.985083 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:29.011949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:29.011890 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" podStartSLOduration=2.274298836 podStartE2EDuration="7.01187055s" totalImagesPullingTime="3.559199416s" totalInitContainerRuntime="1.178372298s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:22 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:29.009252915 +0000 UTC m=+379.025624267" watchObservedRunningTime="2026-07-15 16:43:29.01187055 +0000 UTC m=+379.028241901" Jul 15 16:43:32.402352 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:32.402292 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:32.493196 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:32.493164 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:39.311879 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:39.311845 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:39.412068 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:39.412033 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:39.992026 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:39.991998 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-67f48c5d8-4nl2s" Jul 15 16:43:44.305443 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:44.305404 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:44.435037 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:44.435004 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:45.062448 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.062413 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:45.155820 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.155791 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs"] Jul 15 16:43:45.163078 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.163058 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.166545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.166522 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jul 15 16:43:45.166666 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.166563 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-g2ngq\"" Jul 15 16:43:45.167440 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.167424 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jul 15 16:43:45.171627 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.171606 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs"] Jul 15 16:43:45.291838 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.291802 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tmp\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.292024 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.291848 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tls-certs\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.292024 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.291919 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kvjmn\" (UniqueName: \"kubernetes.io/projected/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-kube-api-access-kvjmn\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.297197 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.297168 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:45.393306 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.393210 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tmp\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.393306 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.393246 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tls-certs\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.393306 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.393281 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-kvjmn\" (UniqueName: \"kubernetes.io/projected/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-kube-api-access-kvjmn\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.395889 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.395861 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tls-certs\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.396229 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.396212 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-tmp\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.405463 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.405435 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-kvjmn\" (UniqueName: \"kubernetes.io/projected/806e589e-a2cc-4f84-9bf8-b9bca4de08f3-kube-api-access-kvjmn\") pod \"kube-auth-proxy-7df86594bb-b4dvs\" (UID: \"806e589e-a2cc-4f84-9bf8-b9bca4de08f3\") " pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.405975 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.405945 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:45.473551 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.473523 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" Jul 15 16:43:45.501664 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.501635 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:45.629442 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.629407 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs"] Jul 15 16:43:45.633524 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:45.633497 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod806e589e_a2cc_4f84_9bf8_b9bca4de08f3.slice/crio-79607ded880746538f863999d5ba99916f8cf8ea4734acd25375a7025dea1741 WatchSource:0}: Error finding container 79607ded880746538f863999d5ba99916f8cf8ea4734acd25375a7025dea1741: Status 404 returned error can't find the container with id 79607ded880746538f863999d5ba99916f8cf8ea4734acd25375a7025dea1741 Jul 15 16:43:45.657119 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:45.657020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:46.034705 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.034643 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:46.063559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.063524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" event={"ID":"806e589e-a2cc-4f84-9bf8-b9bca4de08f3","Type":"ContainerStarted","Data":"79607ded880746538f863999d5ba99916f8cf8ea4734acd25375a7025dea1741"} Jul 15 16:43:46.101478 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.101445 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp54s"] Jul 15 16:43:46.105295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.105275 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.110921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.110898 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-tls\"" Jul 15 16:43:46.110921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.110898 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"model-serving-api-dockercfg-twjkv\"" Jul 15 16:43:46.187318 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.187280 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:46.187931 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.187906 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp54s"] Jul 15 16:43:46.200621 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.200591 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mbht5\" (UniqueName: \"kubernetes.io/projected/7225bdf8-a4c5-4b07-b948-10077dbb206d-kube-api-access-mbht5\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.200763 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.200715 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7225bdf8-a4c5-4b07-b948-10077dbb206d-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.302043 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.301960 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7225bdf8-a4c5-4b07-b948-10077dbb206d-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.302043 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.302008 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-mbht5\" (UniqueName: \"kubernetes.io/projected/7225bdf8-a4c5-4b07-b948-10077dbb206d-kube-api-access-mbht5\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.304748 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.304725 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/7225bdf8-a4c5-4b07-b948-10077dbb206d-tls-certs\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.323617 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.323587 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-mbht5\" (UniqueName: \"kubernetes.io/projected/7225bdf8-a4c5-4b07-b948-10077dbb206d-kube-api-access-mbht5\") pod \"model-serving-api-54d89bd5fc-gp54s\" (UID: \"7225bdf8-a4c5-4b07-b948-10077dbb206d\") " pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.416166 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.416130 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:46.592062 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:46.589755 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/model-serving-api-54d89bd5fc-gp54s"] Jul 15 16:43:46.595568 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:46.595526 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7225bdf8_a4c5_4b07_b948_10077dbb206d.slice/crio-31cb4efb90214f1e9bf2df27533b1a14008e4babc580180ce99f7c791dad022f WatchSource:0}: Error finding container 31cb4efb90214f1e9bf2df27533b1a14008e4babc580180ce99f7c791dad022f: Status 404 returned error can't find the container with id 31cb4efb90214f1e9bf2df27533b1a14008e4babc580180ce99f7c791dad022f Jul 15 16:43:47.068565 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.068524 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" event={"ID":"7225bdf8-a4c5-4b07-b948-10077dbb206d","Type":"ContainerStarted","Data":"31cb4efb90214f1e9bf2df27533b1a14008e4babc580180ce99f7c791dad022f"} Jul 15 16:43:47.553680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.553619 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:47.562924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.562893 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:43:47.565908 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.565888 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.569852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.569832 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\"" Jul 15 16:43:47.569963 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.569832 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"cacerts\"" Jul 15 16:43:47.570293 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.570275 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-openshift-gateway-dockercfg-dl8lr\"" Jul 15 16:43:47.570974 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.570943 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istiod-tls\"" Jul 15 16:43:47.570974 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.570947 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"istio-kubeconfig\"" Jul 15 16:43:47.580126 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.580104 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:43:47.652800 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.652768 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:47.716674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716628 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.716674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716671 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.716925 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716748 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.716925 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716808 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.716925 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716842 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.717093 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.716936 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.717199 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.717094 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-45zl5\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.817928 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.817973 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818018 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818066 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818095 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818145 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.818241 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818185 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-45zl5\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.819503 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.818918 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.821303 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.821233 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.821444 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.821356 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.821810 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.821784 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.823121 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.823090 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.827537 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.827506 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.828444 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.828404 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-45zl5\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5\") pod \"istiod-openshift-gateway-7574459656-x4xvt\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:47.877223 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:47.877189 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:48.045428 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:48.045395 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:43:48.046956 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:48.046927 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod77888647_53c9_4030_9178_5105e481e842.slice/crio-23dfa3cae718094bd424697bcc1a68b21c62d59fbb38afb84b8755918cd2d7e0 WatchSource:0}: Error finding container 23dfa3cae718094bd424697bcc1a68b21c62d59fbb38afb84b8755918cd2d7e0: Status 404 returned error can't find the container with id 23dfa3cae718094bd424697bcc1a68b21c62d59fbb38afb84b8755918cd2d7e0 Jul 15 16:43:48.074297 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:48.074225 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" event={"ID":"77888647-53c9-4030-9178-5105e481e842","Type":"ContainerStarted","Data":"23dfa3cae718094bd424697bcc1a68b21c62d59fbb38afb84b8755918cd2d7e0"} Jul 15 16:43:48.753887 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:48.753848 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:50.088363 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:50.088285 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" event={"ID":"7225bdf8-a4c5-4b07-b948-10077dbb206d","Type":"ContainerStarted","Data":"e7c8443a43ece1b485dfb00b6321b02484c6ceadbc183cb116af01cc68bad604"} Jul 15 16:43:50.093468 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:50.093430 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" event={"ID":"806e589e-a2cc-4f84-9bf8-b9bca4de08f3","Type":"ContainerStarted","Data":"b38aa276cb06f0552ced07d0f2bb59a0e07e15c469e7a0cccff8b6c824e49306"} Jul 15 16:43:50.111907 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:50.111859 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" podStartSLOduration=1.5676124310000001 podStartE2EDuration="4.111843946s" totalImagesPullingTime="2.544231515s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:50.109346381 +0000 UTC m=+400.125717730" watchObservedRunningTime="2026-07-15 16:43:50.111843946 +0000 UTC m=+400.128215300" Jul 15 16:43:50.143026 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:50.142967 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-7df86594bb-b4dvs" podStartSLOduration=1.634868585 podStartE2EDuration="5.142944823s" totalImagesPullingTime="3.508076238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:50.140198983 +0000 UTC m=+400.156570347" watchObservedRunningTime="2026-07-15 16:43:50.142944823 +0000 UTC m=+400.159316178" Jul 15 16:43:51.098199 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:51.098163 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:43:51.169168 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:51.169122 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 16:43:51.169298 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:51.169215 2566 kubelet_resources.go:64] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236220Ki","pods":"250"} Jul 15 16:43:51.895839 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:51.895787 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:51.999250 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:51.999214 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:52.104238 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:52.104197 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" event={"ID":"77888647-53c9-4030-9178-5105e481e842","Type":"ContainerStarted","Data":"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74"} Jul 15 16:43:52.273963 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:52.273932 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:52.427488 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:52.427453 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:52.538031 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:52.537947 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:53.112411 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:53.112372 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:53.114431 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:53.114401 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:43:53.140977 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:53.140932 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" podStartSLOduration=3.021978537 podStartE2EDuration="6.140918398s" totalImagesPullingTime="3.118939861s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:47 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:43:52.156366053 +0000 UTC m=+402.172737405" watchObservedRunningTime="2026-07-15 16:43:53.140918398 +0000 UTC m=+403.157289748" Jul 15 16:43:56.976569 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.976537 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p"] Jul 15 16:43:56.980066 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.980047 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:56.981760 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.981732 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:56.988319 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.988291 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-webhook-server-cert\"" Jul 15 16:43:56.988464 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.988445 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"kserve-controller-manager-dockercfg-ggfdf\"" Jul 15 16:43:56.994052 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.994025 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bfq7c\" (UniqueName: \"kubernetes.io/projected/b57c744b-2dc0-439a-8fff-056e5c921e4e-kube-api-access-bfq7c\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:56.994174 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:56.994144 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57c744b-2dc0-439a-8fff-056e5c921e4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.007181 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.007132 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p"] Jul 15 16:43:57.031376 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.031342 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh"] Jul 15 16:43:57.034800 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.034784 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.038882 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.038860 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-webhook-server-cert\"" Jul 15 16:43:57.041854 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.041838 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"llmisvc-controller-manager-dockercfg-ldfs9\"" Jul 15 16:43:57.050010 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.049982 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh"] Jul 15 16:43:57.094621 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.094590 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bfq7c\" (UniqueName: \"kubernetes.io/projected/b57c744b-2dc0-439a-8fff-056e5c921e4e-kube-api-access-bfq7c\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.094803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.094660 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/141cee3a-4464-4281-9b50-56f4a43d94aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.094803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.094681 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57c744b-2dc0-439a-8fff-056e5c921e4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.094803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.094713 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j59h5\" (UniqueName: \"kubernetes.io/projected/141cee3a-4464-4281-9b50-56f4a43d94aa-kube-api-access-j59h5\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.097545 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.097521 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/b57c744b-2dc0-439a-8fff-056e5c921e4e-cert\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.118569 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.118531 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-bfq7c\" (UniqueName: \"kubernetes.io/projected/b57c744b-2dc0-439a-8fff-056e5c921e4e-kube-api-access-bfq7c\") pod \"kserve-controller-manager-d6f4b65b8-fnj4p\" (UID: \"b57c744b-2dc0-439a-8fff-056e5c921e4e\") " pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.148702 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.148668 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:57.195836 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.195798 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/141cee3a-4464-4281-9b50-56f4a43d94aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.196014 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.195846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j59h5\" (UniqueName: \"kubernetes.io/projected/141cee3a-4464-4281-9b50-56f4a43d94aa-kube-api-access-j59h5\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.198436 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.198406 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/141cee3a-4464-4281-9b50-56f4a43d94aa-cert\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.214748 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.214722 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-j59h5\" (UniqueName: \"kubernetes.io/projected/141cee3a-4464-4281-9b50-56f4a43d94aa-kube-api-access-j59h5\") pod \"llmisvc-controller-manager-777c6bd7d5-ncsmh\" (UID: \"141cee3a-4464-4281-9b50-56f4a43d94aa\") " pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.291277 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.291193 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:43:57.344718 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.344684 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:43:57.447672 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.447376 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p"] Jul 15 16:43:57.452021 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:57.451983 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb57c744b_2dc0_439a_8fff_056e5c921e4e.slice/crio-ae496acd3a1528f148d5ff812a8f1350db30c0e28ccb8c4693b7656d4cfc281f WatchSource:0}: Error finding container ae496acd3a1528f148d5ff812a8f1350db30c0e28ccb8c4693b7656d4cfc281f: Status 404 returned error can't find the container with id ae496acd3a1528f148d5ff812a8f1350db30c0e28ccb8c4693b7656d4cfc281f Jul 15 16:43:57.526686 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:57.526655 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh"] Jul 15 16:43:57.527758 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:43:57.527728 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod141cee3a_4464_4281_9b50_56f4a43d94aa.slice/crio-ddc6abc111b224b02371d30919871003ddf10eb244f8ed2502d876534d9d6652 WatchSource:0}: Error finding container ddc6abc111b224b02371d30919871003ddf10eb244f8ed2502d876534d9d6652: Status 404 returned error can't find the container with id ddc6abc111b224b02371d30919871003ddf10eb244f8ed2502d876534d9d6652 Jul 15 16:43:58.135347 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:58.135301 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" event={"ID":"141cee3a-4464-4281-9b50-56f4a43d94aa","Type":"ContainerStarted","Data":"ddc6abc111b224b02371d30919871003ddf10eb244f8ed2502d876534d9d6652"} Jul 15 16:43:58.136827 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:58.136785 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" event={"ID":"b57c744b-2dc0-439a-8fff-056e5c921e4e","Type":"ContainerStarted","Data":"ae496acd3a1528f148d5ff812a8f1350db30c0e28ccb8c4693b7656d4cfc281f"} Jul 15 16:43:58.436765 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:58.436678 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:43:58.538766 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:43:58.538682 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:44:01.153302 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.153265 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" event={"ID":"b57c744b-2dc0-439a-8fff-056e5c921e4e","Type":"ContainerStarted","Data":"52ef705d6d658cb69dd2e09e81bfb646382497d7a940d2a10d91cca4bcf4e274"} Jul 15 16:44:01.153741 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.153455 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:44:01.154888 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.154861 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" event={"ID":"141cee3a-4464-4281-9b50-56f4a43d94aa","Type":"ContainerStarted","Data":"bf4be576c72bd81791d304f9d92575bb2bd92e27f3fedd0ee9f195491593e204"} Jul 15 16:44:01.154991 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.154918 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:44:01.183645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.183601 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" podStartSLOduration=1.689813874 podStartE2EDuration="5.183587508s" totalImagesPullingTime="3.493773634s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:44:01.182223911 +0000 UTC m=+411.198595264" watchObservedRunningTime="2026-07-15 16:44:01.183587508 +0000 UTC m=+411.199958860" Jul 15 16:44:01.220501 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:01.220456 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" podStartSLOduration=1.798595503 podStartE2EDuration="5.220440835s" totalImagesPullingTime="3.421845332s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:43:56 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:44:01.217744896 +0000 UTC m=+411.234116247" watchObservedRunningTime="2026-07-15 16:44:01.220440835 +0000 UTC m=+411.236812186" Jul 15 16:44:02.108683 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:02.108653 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/model-serving-api-54d89bd5fc-gp54s" Jul 15 16:44:02.152526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:02.152495 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:44:04.717630 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:04.717597 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:44:04.718023 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:04.717786 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" podUID="77888647-53c9-4030-9178-5105e481e842" containerName="discovery" containerID="cri-o://b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74" gracePeriod=30 Jul 15 16:44:04.969878 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:04.969822 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:44:05.067190 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067157 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067203 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067230 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067286 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067311 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067354 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-45zl5\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067380 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts\") pod \"77888647-53c9-4030-9178-5105e481e842\" (UID: \"77888647-53c9-4030-9178-5105e481e842\") " Jul 15 16:44:05.067838 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.067787 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap" (OuterVolumeSpecName: "istio-csr-ca-configmap") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "istio-csr-ca-configmap". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 15 16:44:05.069725 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.069699 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts" (OuterVolumeSpecName: "cacerts") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "cacerts". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:44:05.069833 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.069735 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5" (OuterVolumeSpecName: "kube-api-access-45zl5") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "kube-api-access-45zl5". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:44:05.069929 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.069910 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert" (OuterVolumeSpecName: "istio-csr-dns-cert") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "istio-csr-dns-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:44:05.069997 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.069977 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs" (OuterVolumeSpecName: "local-certs") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "local-certs". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 16:44:05.070047 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.069985 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig" (OuterVolumeSpecName: "istio-kubeconfig") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "istio-kubeconfig". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 15 16:44:05.070098 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.070069 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token" (OuterVolumeSpecName: "istio-token") pod "77888647-53c9-4030-9178-5105e481e842" (UID: "77888647-53c9-4030-9178-5105e481e842"). InnerVolumeSpecName "istio-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:44:05.168647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168624 2566 reconciler_common.go:299] "Volume detached for volume \"istio-kubeconfig\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-kubeconfig\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168644 2566 reconciler_common.go:299] "Volume detached for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-istio-token\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168654 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-dns-cert\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-istio-csr-dns-cert\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168662 2566 reconciler_common.go:299] "Volume detached for volume \"local-certs\" (UniqueName: \"kubernetes.io/empty-dir/77888647-53c9-4030-9178-5105e481e842-local-certs\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168670 2566 reconciler_common.go:299] "Volume detached for volume \"istio-csr-ca-configmap\" (UniqueName: \"kubernetes.io/configmap/77888647-53c9-4030-9178-5105e481e842-istio-csr-ca-configmap\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168679 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-45zl5\" (UniqueName: \"kubernetes.io/projected/77888647-53c9-4030-9178-5105e481e842-kube-api-access-45zl5\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.168803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.168687 2566 reconciler_common.go:299] "Volume detached for volume \"cacerts\" (UniqueName: \"kubernetes.io/secret/77888647-53c9-4030-9178-5105e481e842-cacerts\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:05.174477 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.174449 2566 generic.go:356] "Generic (PLEG): container finished" podID="77888647-53c9-4030-9178-5105e481e842" containerID="b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74" exitCode=0 Jul 15 16:44:05.174565 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.174492 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" Jul 15 16:44:05.174565 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.174536 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" event={"ID":"77888647-53c9-4030-9178-5105e481e842","Type":"ContainerDied","Data":"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74"} Jul 15 16:44:05.174644 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.174591 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt" event={"ID":"77888647-53c9-4030-9178-5105e481e842","Type":"ContainerDied","Data":"23dfa3cae718094bd424697bcc1a68b21c62d59fbb38afb84b8755918cd2d7e0"} Jul 15 16:44:05.174644 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.174613 2566 scope.go:117] "RemoveContainer" containerID="b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74" Jul 15 16:44:05.183202 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.183185 2566 scope.go:117] "RemoveContainer" containerID="b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74" Jul 15 16:44:05.183523 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:44:05.183504 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74\": container with ID starting with b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74 not found: ID does not exist" containerID="b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74" Jul 15 16:44:05.183584 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.183530 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74"} err="failed to get container status \"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74\": rpc error: code = NotFound desc = could not find container \"b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74\": container with ID starting with b5d1e78da5bdef192055891725ef6241805657ecbbe9fd44bb93329e5130cf74 not found: ID does not exist" Jul 15 16:44:05.200674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.200649 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:44:05.203109 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:05.203084 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-ingress/istiod-openshift-gateway-7574459656-x4xvt"] Jul 15 16:44:06.522766 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:06.522735 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="77888647-53c9-4030-9178-5105e481e842" path="/var/lib/kubelet/pods/77888647-53c9-4030-9178-5105e481e842/volumes" Jul 15 16:44:32.161792 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:32.161710 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/llmisvc-controller-manager-777c6bd7d5-ncsmh" Jul 15 16:44:32.164758 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:32.164739 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/kserve-controller-manager-d6f4b65b8-fnj4p" Jul 15 16:44:40.747557 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:40.747518 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:44:45.884594 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.884565 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:45.884957 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.884900 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="77888647-53c9-4030-9178-5105e481e842" containerName="discovery" Jul 15 16:44:45.884957 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.884910 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="77888647-53c9-4030-9178-5105e481e842" containerName="discovery" Jul 15 16:44:45.885050 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.884986 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="77888647-53c9-4030-9178-5105e481e842" containerName="discovery" Jul 15 16:44:45.886830 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.886813 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:45.891535 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.891505 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jul 15 16:44:45.891657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.891641 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-2g8b5\"" Jul 15 16:44:45.892674 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.892630 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jul 15 16:44:45.903697 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.903669 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:45.906369 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:45.906319 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9z795\" (UniqueName: \"kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795\") pod \"kuadrant-operator-catalog-hm9rv\" (UID: \"2a785c14-74eb-40f8-9c36-ff096a7a955c\") " pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:46.006833 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.006799 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9z795\" (UniqueName: \"kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795\") pod \"kuadrant-operator-catalog-hm9rv\" (UID: \"2a785c14-74eb-40f8-9c36-ff096a7a955c\") " pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:46.015710 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.015683 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9z795\" (UniqueName: \"kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795\") pod \"kuadrant-operator-catalog-hm9rv\" (UID: \"2a785c14-74eb-40f8-9c36-ff096a7a955c\") " pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:46.196160 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.196082 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:46.221426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.221397 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:46.328232 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.328201 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:46.331252 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:44:46.331221 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2a785c14_74eb_40f8_9c36_ff096a7a955c.slice/crio-288585e8e0c68e4707bf267cf5dda487774afbecfc16e51f389b1c954a0b1af7 WatchSource:0}: Error finding container 288585e8e0c68e4707bf267cf5dda487774afbecfc16e51f389b1c954a0b1af7: Status 404 returned error can't find the container with id 288585e8e0c68e4707bf267cf5dda487774afbecfc16e51f389b1c954a0b1af7 Jul 15 16:44:46.362803 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.362775 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" event={"ID":"2a785c14-74eb-40f8-9c36-ff096a7a955c","Type":"ContainerStarted","Data":"288585e8e0c68e4707bf267cf5dda487774afbecfc16e51f389b1c954a0b1af7"} Jul 15 16:44:46.431579 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.431554 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfb8j"] Jul 15 16:44:46.434421 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.434401 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:46.449716 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.449652 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfb8j"] Jul 15 16:44:46.511926 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.511896 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fcg6h\" (UniqueName: \"kubernetes.io/projected/57ec28fe-92d0-483f-bbbe-912ec1464213-kube-api-access-fcg6h\") pod \"kuadrant-operator-catalog-hfb8j\" (UID: \"57ec28fe-92d0-483f-bbbe-912ec1464213\") " pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:46.612625 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.612407 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fcg6h\" (UniqueName: \"kubernetes.io/projected/57ec28fe-92d0-483f-bbbe-912ec1464213-kube-api-access-fcg6h\") pod \"kuadrant-operator-catalog-hfb8j\" (UID: \"57ec28fe-92d0-483f-bbbe-912ec1464213\") " pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:46.621530 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.621510 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-fcg6h\" (UniqueName: \"kubernetes.io/projected/57ec28fe-92d0-483f-bbbe-912ec1464213-kube-api-access-fcg6h\") pod \"kuadrant-operator-catalog-hfb8j\" (UID: \"57ec28fe-92d0-483f-bbbe-912ec1464213\") " pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:46.743628 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.743591 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:46.871652 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:46.871619 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hfb8j"] Jul 15 16:44:46.873247 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:44:46.873224 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod57ec28fe_92d0_483f_bbbe_912ec1464213.slice/crio-1c580b40d6b46b641ea10313c7e30c30e0aef874df90d2911a41bc60fb54f919 WatchSource:0}: Error finding container 1c580b40d6b46b641ea10313c7e30c30e0aef874df90d2911a41bc60fb54f919: Status 404 returned error can't find the container with id 1c580b40d6b46b641ea10313c7e30c30e0aef874df90d2911a41bc60fb54f919 Jul 15 16:44:47.368181 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:47.368147 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" event={"ID":"57ec28fe-92d0-483f-bbbe-912ec1464213","Type":"ContainerStarted","Data":"1c580b40d6b46b641ea10313c7e30c30e0aef874df90d2911a41bc60fb54f919"} Jul 15 16:44:47.432501 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:47.432466 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:44:50.386267 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.386184 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" event={"ID":"2a785c14-74eb-40f8-9c36-ff096a7a955c","Type":"ContainerStarted","Data":"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434"} Jul 15 16:44:50.386267 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.386191 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" podUID="2a785c14-74eb-40f8-9c36-ff096a7a955c" containerName="registry-server" containerID="cri-o://bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434" gracePeriod=2 Jul 15 16:44:50.387662 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.387634 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" event={"ID":"57ec28fe-92d0-483f-bbbe-912ec1464213","Type":"ContainerStarted","Data":"ff56628598d9b41254db26e30f2fc0ee6ace54432f6ba9354f77afff82d48015"} Jul 15 16:44:50.415455 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.415409 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" podStartSLOduration=1.845280526 podStartE2EDuration="5.415395595s" totalImagesPullingTime="3.570115069s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:44:45 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:44:50.415166318 +0000 UTC m=+460.431537668" watchObservedRunningTime="2026-07-15 16:44:50.415395595 +0000 UTC m=+460.431766941" Jul 15 16:44:50.439483 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.439440 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" podStartSLOduration=1.331144978 podStartE2EDuration="4.439428216s" totalImagesPullingTime="3.108283238s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:44:46 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:44:50.439054913 +0000 UTC m=+460.455426264" watchObservedRunningTime="2026-07-15 16:44:50.439428216 +0000 UTC m=+460.455799569" Jul 15 16:44:50.631203 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.631181 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:50.648303 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.648250 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9z795\" (UniqueName: \"kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795\") pod \"2a785c14-74eb-40f8-9c36-ff096a7a955c\" (UID: \"2a785c14-74eb-40f8-9c36-ff096a7a955c\") " Jul 15 16:44:50.650558 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.650531 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795" (OuterVolumeSpecName: "kube-api-access-9z795") pod "2a785c14-74eb-40f8-9c36-ff096a7a955c" (UID: "2a785c14-74eb-40f8-9c36-ff096a7a955c"). InnerVolumeSpecName "kube-api-access-9z795". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:44:50.749215 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:50.749185 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9z795\" (UniqueName: \"kubernetes.io/projected/2a785c14-74eb-40f8-9c36-ff096a7a955c-kube-api-access-9z795\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:44:51.393837 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.393801 2566 generic.go:356] "Generic (PLEG): container finished" podID="2a785c14-74eb-40f8-9c36-ff096a7a955c" containerID="bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434" exitCode=0 Jul 15 16:44:51.394294 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.393847 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" Jul 15 16:44:51.394294 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.393886 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" event={"ID":"2a785c14-74eb-40f8-9c36-ff096a7a955c","Type":"ContainerDied","Data":"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434"} Jul 15 16:44:51.394294 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.393936 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-hm9rv" event={"ID":"2a785c14-74eb-40f8-9c36-ff096a7a955c","Type":"ContainerDied","Data":"288585e8e0c68e4707bf267cf5dda487774afbecfc16e51f389b1c954a0b1af7"} Jul 15 16:44:51.394294 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.393956 2566 scope.go:117] "RemoveContainer" containerID="bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434" Jul 15 16:44:51.403612 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.403596 2566 scope.go:117] "RemoveContainer" containerID="bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434" Jul 15 16:44:51.403870 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:44:51.403852 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434\": container with ID starting with bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434 not found: ID does not exist" containerID="bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434" Jul 15 16:44:51.403927 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.403883 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434"} err="failed to get container status \"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434\": rpc error: code = NotFound desc = could not find container \"bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434\": container with ID starting with bcd628079db8fff267d9da8f945b9b915eb7ba508147ecb935add2bad5cac434 not found: ID does not exist" Jul 15 16:44:51.416404 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.416382 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:51.418538 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:51.418520 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-hm9rv"] Jul 15 16:44:52.522789 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:52.522757 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2a785c14-74eb-40f8-9c36-ff096a7a955c" path="/var/lib/kubelet/pods/2a785c14-74eb-40f8-9c36-ff096a7a955c/volumes" Jul 15 16:44:56.744027 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:56.743992 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:56.744453 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:56.744080 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:56.766543 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:56.766519 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:44:57.443548 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:44:57.443519 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-hfb8j" Jul 15 16:45:05.462305 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:05.462269 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:05.527372 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:05.527311 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:07.295307 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:07.295271 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:12.123791 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:12.123745 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:15.210661 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:15.210628 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:15.824249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:15.824217 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:18.476612 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:18.476577 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:19.279680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.279650 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtj99"] Jul 15 16:45:19.280025 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.280012 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2a785c14-74eb-40f8-9c36-ff096a7a955c" containerName="registry-server" Jul 15 16:45:19.280071 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.280026 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="2a785c14-74eb-40f8-9c36-ff096a7a955c" containerName="registry-server" Jul 15 16:45:19.280105 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.280079 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="2a785c14-74eb-40f8-9c36-ff096a7a955c" containerName="registry-server" Jul 15 16:45:19.284020 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.284002 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:19.286853 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.286837 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-qwxbp\"" Jul 15 16:45:19.297974 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.297955 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtj99"] Jul 15 16:45:19.385047 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.385017 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zpcvg\" (UniqueName: \"kubernetes.io/projected/fa01296e-388b-4ed4-8c72-898216c99193-kube-api-access-zpcvg\") pod \"authorino-operator-75cc8d5cb7-wtj99\" (UID: \"fa01296e-388b-4ed4-8c72-898216c99193\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:19.485841 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.485812 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-zpcvg\" (UniqueName: \"kubernetes.io/projected/fa01296e-388b-4ed4-8c72-898216c99193-kube-api-access-zpcvg\") pod \"authorino-operator-75cc8d5cb7-wtj99\" (UID: \"fa01296e-388b-4ed4-8c72-898216c99193\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:19.501980 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.501946 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-zpcvg\" (UniqueName: \"kubernetes.io/projected/fa01296e-388b-4ed4-8c72-898216c99193-kube-api-access-zpcvg\") pod \"authorino-operator-75cc8d5cb7-wtj99\" (UID: \"fa01296e-388b-4ed4-8c72-898216c99193\") " pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:19.594633 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.594554 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:19.734754 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:19.734724 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-75cc8d5cb7-wtj99"] Jul 15 16:45:19.738199 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:19.738161 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa01296e_388b_4ed4_8c72_898216c99193.slice/crio-d76ba630d75968eed74ba12da784b603f19c84294b2011fc93985fb8679fc268 WatchSource:0}: Error finding container d76ba630d75968eed74ba12da784b603f19c84294b2011fc93985fb8679fc268: Status 404 returned error can't find the container with id d76ba630d75968eed74ba12da784b603f19c84294b2011fc93985fb8679fc268 Jul 15 16:45:20.053771 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:20.053741 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:20.177735 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:20.177703 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:20.526165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:20.526128 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" event={"ID":"fa01296e-388b-4ed4-8c72-898216c99193","Type":"ContainerStarted","Data":"d76ba630d75968eed74ba12da784b603f19c84294b2011fc93985fb8679fc268"} Jul 15 16:45:22.538006 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:22.537972 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" event={"ID":"fa01296e-388b-4ed4-8c72-898216c99193","Type":"ContainerStarted","Data":"f73e4de3f5582d1865d9664cc39b525b4f2f7399319a244e5baa07c38980c668"} Jul 15 16:45:22.538383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:22.538036 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:22.608983 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:22.608926 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" podStartSLOduration=1.23821248 podStartE2EDuration="3.608910984s" totalImagesPullingTime="2.370698504s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:22.608005172 +0000 UTC m=+492.624376525" watchObservedRunningTime="2026-07-15 16:45:22.608910984 +0000 UTC m=+492.625282335" Jul 15 16:45:23.797779 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.797748 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh"] Jul 15 16:45:23.801090 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.801073 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:23.803962 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.803937 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4mv6t\"" Jul 15 16:45:23.818209 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.818181 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh"] Jul 15 16:45:23.924667 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.924633 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d8pwc\" (UniqueName: \"kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:23.924819 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:23.924672 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.025968 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.025923 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.026106 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.026025 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-d8pwc\" (UniqueName: \"kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.026308 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.026290 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.035540 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.035512 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-d8pwc\" (UniqueName: \"kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc\") pod \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.110972 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.110900 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:24.245978 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.245950 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh"] Jul 15 16:45:24.247808 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:24.247771 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod099eb0e1_3c98_406f_ad66_1ac8217d8d05.slice/crio-9e14b1e3270df535a03bab1bd85d0ded3ca57f7bac48adab35e88c2f07f70e34 WatchSource:0}: Error finding container 9e14b1e3270df535a03bab1bd85d0ded3ca57f7bac48adab35e88c2f07f70e34: Status 404 returned error can't find the container with id 9e14b1e3270df535a03bab1bd85d0ded3ca57f7bac48adab35e88c2f07f70e34 Jul 15 16:45:24.465978 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.465948 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:24.548192 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.548160 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" event={"ID":"099eb0e1-3c98-406f-ad66-1ac8217d8d05","Type":"ContainerStarted","Data":"9e14b1e3270df535a03bab1bd85d0ded3ca57f7bac48adab35e88c2f07f70e34"} Jul 15 16:45:24.670564 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:24.670531 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:26.119721 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.119687 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm"] Jul 15 16:45:26.124512 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.124496 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:26.127198 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.127174 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"limitador-operator-controller-manager-dockercfg-t2tvf\"" Jul 15 16:45:26.134127 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.134101 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm"] Jul 15 16:45:26.247396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.247366 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn8c6\" (UniqueName: \"kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6\") pod \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" (UID: \"22de6faf-3ac4-4522-9f15-9120d0ea7296\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:26.347984 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.347953 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wn8c6\" (UniqueName: \"kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6\") pod \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" (UID: \"22de6faf-3ac4-4522-9f15-9120d0ea7296\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:26.357262 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.357237 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn8c6\" (UniqueName: \"kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6\") pod \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" (UID: \"22de6faf-3ac4-4522-9f15-9120d0ea7296\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:26.435581 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.435508 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:26.585207 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.585183 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm"] Jul 15 16:45:26.587217 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:26.587190 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod22de6faf_3ac4_4522_9f15_9120d0ea7296.slice/crio-77e4d2a72f08a73c48d6b6a938c53029e54cf715141e95cd79c88030dfa236b0 WatchSource:0}: Error finding container 77e4d2a72f08a73c48d6b6a938c53029e54cf715141e95cd79c88030dfa236b0: Status 404 returned error can't find the container with id 77e4d2a72f08a73c48d6b6a938c53029e54cf715141e95cd79c88030dfa236b0 Jul 15 16:45:26.622908 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.622871 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:26.901669 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:26.901616 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:27.569761 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:27.569719 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" event={"ID":"22de6faf-3ac4-4522-9f15-9120d0ea7296","Type":"ContainerStarted","Data":"77e4d2a72f08a73c48d6b6a938c53029e54cf715141e95cd79c88030dfa236b0"} Jul 15 16:45:28.359606 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.359569 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr"] Jul 15 16:45:28.363943 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.363923 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:28.367159 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.367134 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-env\"" Jul 15 16:45:28.367418 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.367403 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"dns-operator-controller-manager-dockercfg-qrvh9\"" Jul 15 16:45:28.375137 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.375108 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr"] Jul 15 16:45:28.465726 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.465687 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pq4zt\" (UniqueName: \"kubernetes.io/projected/0e9d901b-d3a2-447a-beb1-c742ed401f4e-kube-api-access-pq4zt\") pod \"dns-operator-controller-manager-864dd78c4c-gtpcr\" (UID: \"0e9d901b-d3a2-447a-beb1-c742ed401f4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:28.567020 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.566996 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pq4zt\" (UniqueName: \"kubernetes.io/projected/0e9d901b-d3a2-447a-beb1-c742ed401f4e-kube-api-access-pq4zt\") pod \"dns-operator-controller-manager-864dd78c4c-gtpcr\" (UID: \"0e9d901b-d3a2-447a-beb1-c742ed401f4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:28.576177 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.576152 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-pq4zt\" (UniqueName: \"kubernetes.io/projected/0e9d901b-d3a2-447a-beb1-c742ed401f4e-kube-api-access-pq4zt\") pod \"dns-operator-controller-manager-864dd78c4c-gtpcr\" (UID: \"0e9d901b-d3a2-447a-beb1-c742ed401f4e\") " pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:28.677359 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.677319 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:28.816422 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:28.816390 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr"] Jul 15 16:45:28.819073 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:28.819043 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e9d901b_d3a2_447a_beb1_c742ed401f4e.slice/crio-f6008404253b048a5f114943c08be047d755c8c28946b33ecc81f85401bcf504 WatchSource:0}: Error finding container f6008404253b048a5f114943c08be047d755c8c28946b33ecc81f85401bcf504: Status 404 returned error can't find the container with id f6008404253b048a5f114943c08be047d755c8c28946b33ecc81f85401bcf504 Jul 15 16:45:29.584487 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:29.584444 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" event={"ID":"0e9d901b-d3a2-447a-beb1-c742ed401f4e","Type":"ContainerStarted","Data":"f6008404253b048a5f114943c08be047d755c8c28946b33ecc81f85401bcf504"} Jul 15 16:45:29.585961 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:29.585934 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" event={"ID":"22de6faf-3ac4-4522-9f15-9120d0ea7296","Type":"ContainerStarted","Data":"8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a"} Jul 15 16:45:29.586100 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:29.586057 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:29.633687 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:29.633632 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" podStartSLOduration=1.660320343 podStartE2EDuration="3.633616456s" totalImagesPullingTime="1.973296113s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:26 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:29.632627197 +0000 UTC m=+499.648998559" watchObservedRunningTime="2026-07-15 16:45:29.633616456 +0000 UTC m=+499.649987806" Jul 15 16:45:29.722367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:29.722318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:30.012571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:30.012540 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:30.093960 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:30.093927 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:31.597014 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:31.596977 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" event={"ID":"0e9d901b-d3a2-447a-beb1-c742ed401f4e","Type":"ContainerStarted","Data":"ea15f2ca2d35444ee9f4c27d10e62db49b1ebcac7213d53831567ca5ab8e6fd9"} Jul 15 16:45:31.597398 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:31.597134 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:31.615541 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:31.615497 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" podStartSLOduration=1.518905875 podStartE2EDuration="3.615481776s" totalImagesPullingTime="2.096575901s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:28 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:31.614774839 +0000 UTC m=+501.631146191" watchObservedRunningTime="2026-07-15 16:45:31.615481776 +0000 UTC m=+501.631853127" Jul 15 16:45:33.545805 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:33.545771 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-75cc8d5cb7-wtj99" Jul 15 16:45:33.608987 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:33.608939 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" event={"ID":"099eb0e1-3c98-406f-ad66-1ac8217d8d05","Type":"ContainerStarted","Data":"fd93ff5ecc806ba097ad4404f05981f28968950855f6453d6c77522e4945bfbf"} Jul 15 16:45:33.609142 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:33.609088 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:33.632155 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:33.632106 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" podStartSLOduration=1.7397752359999998 podStartE2EDuration="10.632089743s" totalImagesPullingTime="8.892314507s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:23 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:33.630010613 +0000 UTC m=+503.646381965" watchObservedRunningTime="2026-07-15 16:45:33.632089743 +0000 UTC m=+503.648461152" Jul 15 16:45:35.027530 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.027492 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:35.222010 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.221975 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:35.962319 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.962277 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4"] Jul 15 16:45:35.965980 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.965964 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:35.969184 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.969163 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jul 15 16:45:35.969418 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.969394 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jul 15 16:45:35.969546 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.969495 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-pszz2\"" Jul 15 16:45:35.975457 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.975433 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4"] Jul 15 16:45:35.990209 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:35.990180 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:36.033593 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.033568 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.033918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.033650 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nrfxn\" (UniqueName: \"kubernetes.io/projected/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-kube-api-access-nrfxn\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.033918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.033684 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.134274 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.134238 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.134449 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.134348 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nrfxn\" (UniqueName: \"kubernetes.io/projected/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-kube-api-access-nrfxn\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.134449 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.134377 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.134563 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:36.134486 2566 secret.go:189] Couldn't get secret kuadrant-system/plugin-serving-cert: secret "plugin-serving-cert" not found Jul 15 16:45:36.134620 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:36.134584 2566 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert podName:f212bb1e-f489-4b2e-8deb-446a26f9d8a4 nodeName:}" failed. No retries permitted until 2026-07-15 16:45:36.6345604 +0000 UTC m=+506.650931743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "plugin-serving-cert" (UniqueName: "kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert") pod "kuadrant-console-plugin-66b5774985-sd6l4" (UID: "f212bb1e-f489-4b2e-8deb-446a26f9d8a4") : secret "plugin-serving-cert" not found Jul 15 16:45:36.134858 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.134836 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-nginx-conf\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.143254 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.143235 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-nrfxn\" (UniqueName: \"kubernetes.io/projected/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-kube-api-access-nrfxn\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.324187 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.324109 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:36.637852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.637780 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.640133 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.640115 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/f212bb1e-f489-4b2e-8deb-446a26f9d8a4-plugin-serving-cert\") pod \"kuadrant-console-plugin-66b5774985-sd6l4\" (UID: \"f212bb1e-f489-4b2e-8deb-446a26f9d8a4\") " pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:36.875768 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:36.875738 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" Jul 15 16:45:37.006845 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:37.006821 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4"] Jul 15 16:45:37.008893 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:37.008865 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podf212bb1e_f489_4b2e_8deb_446a26f9d8a4.slice/crio-880543ff688cf50fbf659f31f61d14e705d4b5092f48ddc518b04bcce891f608 WatchSource:0}: Error finding container 880543ff688cf50fbf659f31f61d14e705d4b5092f48ddc518b04bcce891f608: Status 404 returned error can't find the container with id 880543ff688cf50fbf659f31f61d14e705d4b5092f48ddc518b04bcce891f608 Jul 15 16:45:37.629028 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:37.628997 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" event={"ID":"f212bb1e-f489-4b2e-8deb-446a26f9d8a4","Type":"ContainerStarted","Data":"880543ff688cf50fbf659f31f61d14e705d4b5092f48ddc518b04bcce891f608"} Jul 15 16:45:40.593901 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:40.593869 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:41.152712 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:41.152677 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:41.265060 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:41.265020 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:42.605039 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:42.605006 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/dns-operator-controller-manager-864dd78c4c-gtpcr" Jul 15 16:45:43.598932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:43.598858 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:43.701521 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:43.701488 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:44.616047 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:44.616015 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.545229 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.545185 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh"] Jul 15 16:45:46.545781 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.545402 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" containerName="manager" containerID="cri-o://fd93ff5ecc806ba097ad4404f05981f28968950855f6453d6c77522e4945bfbf" gracePeriod=2 Jul 15 16:45:46.563199 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.563152 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh"] Jul 15 16:45:46.571898 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.571857 2566 status_manager.go:1041] "Failed to update status for pod" err="failed to patch status \"{\\\"metadata\\\":{\\\"uid\\\":\\\"099eb0e1-3c98-406f-ad66-1ac8217d8d05\\\"},\\\"status\\\":{\\\"$setElementOrder/conditions\\\":[{\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"type\\\":\\\"Initialized\\\"},{\\\"type\\\":\\\"Ready\\\"},{\\\"type\\\":\\\"ContainersReady\\\"},{\\\"type\\\":\\\"PodScheduled\\\"}],\\\"conditions\\\":[{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodReadyToStartContainers\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Initialized\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"Ready\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"ContainersReady\\\"},{\\\"observedGeneration\\\":2,\\\"type\\\":\\\"PodScheduled\\\"}],\\\"observedGeneration\\\":2}}\" for pod \"kuadrant-system\"/\"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\": pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" not found" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.579880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.579852 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm"] Jul 15 16:45:46.580290 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.580080 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" containerName="manager" containerID="cri-o://8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a" gracePeriod=2 Jul 15 16:45:46.584143 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.584108 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.586940 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.586744 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.589542 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.589521 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:45:46.590366 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.590342 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" containerName="manager" Jul 15 16:45:46.590492 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.590369 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" containerName="manager" Jul 15 16:45:46.590558 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.590534 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" containerName="manager" Jul 15 16:45:46.594071 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.594052 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm"] Jul 15 16:45:46.594151 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.594132 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.601228 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.601089 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.620777 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.620708 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:45:46.623359 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.623251 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.623471 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.623369 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hp4dd\" (UniqueName: \"kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.633798 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.633769 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r"] Jul 15 16:45:46.634367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.634316 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" containerName="manager" Jul 15 16:45:46.634367 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.634359 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" containerName="manager" Jul 15 16:45:46.634577 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.634484 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" containerName="manager" Jul 15 16:45:46.637769 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.637751 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:46.655317 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.655154 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.657683 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.657465 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.658539 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.658505 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r"] Jul 15 16:45:46.681638 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.681585 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.683446 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.683418 2566 generic.go:356] "Generic (PLEG): container finished" podID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" containerID="fd93ff5ecc806ba097ad4404f05981f28968950855f6453d6c77522e4945bfbf" exitCode=0 Jul 15 16:45:46.724374 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.724258 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.724374 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.724352 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hp4dd\" (UniqueName: \"kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.724617 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.724391 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-k2mkf\" (UniqueName: \"kubernetes.io/projected/8cf0ff67-f438-4b11-aee3-a4c1d0f61c71-kube-api-access-k2mkf\") pod \"limitador-operator-controller-manager-859d7b78bd-h4g4r\" (UID: \"8cf0ff67-f438-4b11-aee3-a4c1d0f61c71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:46.724752 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.724725 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.734876 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.734847 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hp4dd\" (UniqueName: \"kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd\") pod \"kuadrant-operator-controller-manager-8684c9ff49-gjc5d\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.825796 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.825766 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-k2mkf\" (UniqueName: \"kubernetes.io/projected/8cf0ff67-f438-4b11-aee3-a4c1d0f61c71-kube-api-access-k2mkf\") pod \"limitador-operator-controller-manager-859d7b78bd-h4g4r\" (UID: \"8cf0ff67-f438-4b11-aee3-a4c1d0f61c71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:46.840902 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.840883 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-k2mkf\" (UniqueName: \"kubernetes.io/projected/8cf0ff67-f438-4b11-aee3-a4c1d0f61c71-kube-api-access-k2mkf\") pod \"limitador-operator-controller-manager-859d7b78bd-h4g4r\" (UID: \"8cf0ff67-f438-4b11-aee3-a4c1d0f61c71\") " pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:46.841417 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.841399 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.843777 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.843752 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.844258 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.844243 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.845854 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.845829 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.848392 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.848366 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:46.850305 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:46.850284 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:46.926762 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.926733 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume\") pod \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " Jul 15 16:45:46.926886 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.926871 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d8pwc\" (UniqueName: \"kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc\") pod \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\" (UID: \"099eb0e1-3c98-406f-ad66-1ac8217d8d05\") " Jul 15 16:45:46.926932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.926908 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn8c6\" (UniqueName: \"kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6\") pod \"22de6faf-3ac4-4522-9f15-9120d0ea7296\" (UID: \"22de6faf-3ac4-4522-9f15-9120d0ea7296\") " Jul 15 16:45:46.927270 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.927250 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "099eb0e1-3c98-406f-ad66-1ac8217d8d05" (UID: "099eb0e1-3c98-406f-ad66-1ac8217d8d05"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 16:45:46.928932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.928906 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6" (OuterVolumeSpecName: "kube-api-access-wn8c6") pod "22de6faf-3ac4-4522-9f15-9120d0ea7296" (UID: "22de6faf-3ac4-4522-9f15-9120d0ea7296"). InnerVolumeSpecName "kube-api-access-wn8c6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:45:46.928932 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.928915 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc" (OuterVolumeSpecName: "kube-api-access-d8pwc") pod "099eb0e1-3c98-406f-ad66-1ac8217d8d05" (UID: "099eb0e1-3c98-406f-ad66-1ac8217d8d05"). InnerVolumeSpecName "kube-api-access-d8pwc". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:45:46.983667 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.983634 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:46.991724 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:46.991700 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:47.027723 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.027672 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d8pwc\" (UniqueName: \"kubernetes.io/projected/099eb0e1-3c98-406f-ad66-1ac8217d8d05-kube-api-access-d8pwc\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:45:47.027723 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.027699 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn8c6\" (UniqueName: \"kubernetes.io/projected/22de6faf-3ac4-4522-9f15-9120d0ea7296-kube-api-access-wn8c6\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:45:47.027723 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.027718 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/099eb0e1-3c98-406f-ad66-1ac8217d8d05-extensions-socket-volume\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:45:47.166915 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.166791 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:45:47.169768 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:47.169735 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc404be61_9668_4963_8e10_52aaa9ad578f.slice/crio-6818cfa6ae3b9a70d9b97e2f50ebf63b02df8d4c224459e9ad3ca9657ff58770 WatchSource:0}: Error finding container 6818cfa6ae3b9a70d9b97e2f50ebf63b02df8d4c224459e9ad3ca9657ff58770: Status 404 returned error can't find the container with id 6818cfa6ae3b9a70d9b97e2f50ebf63b02df8d4c224459e9ad3ca9657ff58770 Jul 15 16:45:47.203722 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.203701 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r"] Jul 15 16:45:47.206151 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:45:47.206127 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8cf0ff67_f438_4b11_aee3_a4c1d0f61c71.slice/crio-123d111dabe2c59a515b54a75612fe27aca1ecade076f32d2716efca75a777aa WatchSource:0}: Error finding container 123d111dabe2c59a515b54a75612fe27aca1ecade076f32d2716efca75a777aa: Status 404 returned error can't find the container with id 123d111dabe2c59a515b54a75612fe27aca1ecade076f32d2716efca75a777aa Jul 15 16:45:47.692804 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.692738 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" event={"ID":"8cf0ff67-f438-4b11-aee3-a4c1d0f61c71","Type":"ContainerStarted","Data":"40f31efc91cfee278bcc0454e2d6e28bf736d045f6c1c5fc202d0e222c06a919"} Jul 15 16:45:47.692804 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.692812 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" event={"ID":"8cf0ff67-f438-4b11-aee3-a4c1d0f61c71","Type":"ContainerStarted","Data":"123d111dabe2c59a515b54a75612fe27aca1ecade076f32d2716efca75a777aa"} Jul 15 16:45:47.693429 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.693053 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:45:47.695391 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.695355 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" event={"ID":"c404be61-9668-4963-8e10-52aaa9ad578f","Type":"ContainerStarted","Data":"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac"} Jul 15 16:45:47.695526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.695413 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" event={"ID":"c404be61-9668-4963-8e10-52aaa9ad578f","Type":"ContainerStarted","Data":"6818cfa6ae3b9a70d9b97e2f50ebf63b02df8d4c224459e9ad3ca9657ff58770"} Jul 15 16:45:47.695526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.695443 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:47.695804 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:47.695775 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:47.696921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.696887 2566 generic.go:356] "Generic (PLEG): container finished" podID="22de6faf-3ac4-4522-9f15-9120d0ea7296" containerID="8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a" exitCode=0 Jul 15 16:45:47.697030 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.696945 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:47.697030 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.696991 2566 scope.go:117] "RemoveContainer" containerID="8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a" Jul 15 16:45:47.698595 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:47.698566 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:47.698686 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.698619 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:47.707974 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.707952 2566 scope.go:117] "RemoveContainer" containerID="8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a" Jul 15 16:45:47.708433 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:47.708412 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a\": container with ID starting with 8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a not found: ID does not exist" containerID="8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a" Jul 15 16:45:47.708544 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.708438 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a"} err="failed to get container status \"8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a\": rpc error: code = NotFound desc = could not find container \"8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a\": container with ID starting with 8187db9aee5068174d0b429ca129e40a6adcbda3b2e0a766b1c900722b36232a not found: ID does not exist" Jul 15 16:45:47.708544 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.708458 2566 scope.go:117] "RemoveContainer" containerID="fd93ff5ecc806ba097ad4404f05981f28968950855f6453d6c77522e4945bfbf" Jul 15 16:45:47.734309 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.734258 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" podStartSLOduration=1.734239751 podStartE2EDuration="1.734239751s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:47.733567598 +0000 UTC m=+517.749938949" watchObservedRunningTime="2026-07-15 16:45:47.734239751 +0000 UTC m=+517.750611098" Jul 15 16:45:47.735964 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:47.735937 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"limitador-operator-controller-manager-859d7b78bd-q6lgm\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-q6lgm" Jul 15 16:45:47.762700 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:47.762659 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" podStartSLOduration=1.762643143 podStartE2EDuration="1.762643143s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:46 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:45:47.761199686 +0000 UTC m=+517.777571037" watchObservedRunningTime="2026-07-15 16:45:47.762643143 +0000 UTC m=+517.779014495" Jul 15 16:45:47.763929 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:45:47.763901 2566 status_manager.go:1018] "Failed to get status for pod" err="pods \"kuadrant-operator-controller-manager-8684c9ff49-6fqdh\" is forbidden: User \"system:node:ip-10-0-138-140.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-138-140.ec2.internal' and this object" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-6fqdh" Jul 15 16:45:48.527575 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:48.527544 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="099eb0e1-3c98-406f-ad66-1ac8217d8d05" path="/var/lib/kubelet/pods/099eb0e1-3c98-406f-ad66-1ac8217d8d05/volumes" Jul 15 16:45:48.527981 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:48.527966 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22de6faf-3ac4-4522-9f15-9120d0ea7296" path="/var/lib/kubelet/pods/22de6faf-3ac4-4522-9f15-9120d0ea7296/volumes" Jul 15 16:45:49.322369 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:49.322315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:49.434883 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:49.434844 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:50.009290 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:50.009255 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:50.106462 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:50.106428 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:52.077132 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:52.077091 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:53.508017 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:53.507981 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:54.766727 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:54.766684 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:45:58.710265 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:58.710235 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:45:58.710838 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:45:58.710314 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-operator-controller-manager-859d7b78bd-h4g4r" Jul 15 16:46:00.252521 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:00.252465 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:00.400729 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:00.400673 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:02.592799 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:02.592766 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:02.693133 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:02.693103 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:02.797225 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:02.797189 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" event={"ID":"f212bb1e-f489-4b2e-8deb-446a26f9d8a4","Type":"ContainerStarted","Data":"07d5365e66b86f4227675bf600da2f5d1363b607c24edd943479ce9227697cd7"} Jul 15 16:46:02.816104 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:02.816046 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-66b5774985-sd6l4" podStartSLOduration=2.806041854 podStartE2EDuration="27.816027721s" totalImagesPullingTime="25.009985867s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:45:35 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:46:02.814715007 +0000 UTC m=+532.831086358" watchObservedRunningTime="2026-07-15 16:46:02.816027721 +0000 UTC m=+532.832399073" Jul 15 16:46:03.039180 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.039150 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-6bc974cd7d-h8kkr"] Jul 15 16:46:03.042172 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.042156 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.044991 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.044962 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 15 16:46:03.045138 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.045115 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 15 16:46:03.045220 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.045158 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 15 16:46:03.045278 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.045253 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 15 16:46:03.045278 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.045269 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 15 16:46:03.045406 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.045373 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 15 16:46:03.046424 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.046407 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mtjsr\"" Jul 15 16:46:03.046522 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.046424 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 15 16:46:03.051855 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.051831 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 15 16:46:03.054074 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.054054 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc974cd7d-h8kkr"] Jul 15 16:46:03.110721 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.110696 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:03.180717 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180688 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hpslf\" (UniqueName: \"kubernetes.io/projected/7c98d1cf-42e2-469f-8996-2f676f49db24-kube-api-access-hpslf\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180850 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180720 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-oauth-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180850 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180746 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-service-ca\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180850 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180800 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-oauth-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180853 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-console-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180878 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.180949 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.180898 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-trusted-ca-bundle\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.182810 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.182789 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:03.262785 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.262746 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:03.282052 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282021 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282135 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282058 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-trusted-ca-bundle\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282135 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hpslf\" (UniqueName: \"kubernetes.io/projected/7c98d1cf-42e2-469f-8996-2f676f49db24-kube-api-access-hpslf\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282143 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-oauth-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282176 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-service-ca\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282191 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-oauth-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.282249 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.282209 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-console-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.283040 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.283014 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-service-ca\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.283426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.283015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-console-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.283426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.283015 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-oauth-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.283426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.283189 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7c98d1cf-42e2-469f-8996-2f676f49db24-trusted-ca-bundle\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.284818 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.284782 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-serving-cert\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.284818 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.284799 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/7c98d1cf-42e2-469f-8996-2f676f49db24-console-oauth-config\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.290984 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.290928 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-hpslf\" (UniqueName: \"kubernetes.io/projected/7c98d1cf-42e2-469f-8996-2f676f49db24-kube-api-access-hpslf\") pod \"console-6bc974cd7d-h8kkr\" (UID: \"7c98d1cf-42e2-469f-8996-2f676f49db24\") " pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.353023 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.352996 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:03.491717 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.491688 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-6bc974cd7d-h8kkr"] Jul 15 16:46:03.493827 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:46:03.493801 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c98d1cf_42e2_469f_8996_2f676f49db24.slice/crio-08d2e32cb331e9c729dbc0d5e4f272854106d2de3df89723a5a37e6c4f6205fb WatchSource:0}: Error finding container 08d2e32cb331e9c729dbc0d5e4f272854106d2de3df89723a5a37e6c4f6205fb: Status 404 returned error can't find the container with id 08d2e32cb331e9c729dbc0d5e4f272854106d2de3df89723a5a37e6c4f6205fb Jul 15 16:46:03.804165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.804073 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc974cd7d-h8kkr" event={"ID":"7c98d1cf-42e2-469f-8996-2f676f49db24","Type":"ContainerStarted","Data":"752304119d8810371c4e7fc7f35f52d92250074c26dc54bb929addec17e05a93"} Jul 15 16:46:03.804165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.804132 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-6bc974cd7d-h8kkr" event={"ID":"7c98d1cf-42e2-469f-8996-2f676f49db24","Type":"ContainerStarted","Data":"08d2e32cb331e9c729dbc0d5e4f272854106d2de3df89723a5a37e6c4f6205fb"} Jul 15 16:46:03.827151 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.827100 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-6bc974cd7d-h8kkr" podStartSLOduration=0.827083877 podStartE2EDuration="827.083877ms" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:46:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:46:03.826752396 +0000 UTC m=+533.843123747" watchObservedRunningTime="2026-07-15 16:46:03.827083877 +0000 UTC m=+533.843455228" Jul 15 16:46:03.838464 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.838433 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:03.938882 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:03.938854 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:04.202376 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.202285 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:46:04.202552 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.202524 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" podUID="c404be61-9668-4963-8e10-52aaa9ad578f" containerName="manager" containerID="cri-o://92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac" gracePeriod=10 Jul 15 16:46:04.452705 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.452652 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:46:04.455636 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.455615 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:46:04.456008 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.455997 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c404be61-9668-4963-8e10-52aaa9ad578f" containerName="manager" Jul 15 16:46:04.456052 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.456010 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="c404be61-9668-4963-8e10-52aaa9ad578f" containerName="manager" Jul 15 16:46:04.456098 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.456089 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="c404be61-9668-4963-8e10-52aaa9ad578f" containerName="manager" Jul 15 16:46:04.457913 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.457899 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.476401 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.476378 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:46:04.594598 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.594570 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-hp4dd\" (UniqueName: \"kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd\") pod \"c404be61-9668-4963-8e10-52aaa9ad578f\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " Jul 15 16:46:04.594773 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.594633 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume\") pod \"c404be61-9668-4963-8e10-52aaa9ad578f\" (UID: \"c404be61-9668-4963-8e10-52aaa9ad578f\") " Jul 15 16:46:04.594843 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.594805 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.594939 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.594914 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9v8x9\" (UniqueName: \"kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.595060 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.595042 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "c404be61-9668-4963-8e10-52aaa9ad578f" (UID: "c404be61-9668-4963-8e10-52aaa9ad578f"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 16:46:04.596695 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.596671 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd" (OuterVolumeSpecName: "kube-api-access-hp4dd") pod "c404be61-9668-4963-8e10-52aaa9ad578f" (UID: "c404be61-9668-4963-8e10-52aaa9ad578f"). InnerVolumeSpecName "kube-api-access-hp4dd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:46:04.696210 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.696178 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9v8x9\" (UniqueName: \"kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.696361 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.696245 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.696361 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.696289 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-hp4dd\" (UniqueName: \"kubernetes.io/projected/c404be61-9668-4963-8e10-52aaa9ad578f-kube-api-access-hp4dd\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:46:04.696361 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.696300 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/c404be61-9668-4963-8e10-52aaa9ad578f-extensions-socket-volume\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:46:04.696626 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.696607 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.705395 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.705347 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-9v8x9\" (UniqueName: \"kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9\") pod \"kuadrant-operator-controller-manager-86878c769f-g8lrm\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.767149 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.767121 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:04.810572 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.810546 2566 generic.go:356] "Generic (PLEG): container finished" podID="c404be61-9668-4963-8e10-52aaa9ad578f" containerID="92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac" exitCode=0 Jul 15 16:46:04.810906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.810576 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" event={"ID":"c404be61-9668-4963-8e10-52aaa9ad578f","Type":"ContainerDied","Data":"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac"} Jul 15 16:46:04.810906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.810619 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" Jul 15 16:46:04.810906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.810640 2566 scope.go:117] "RemoveContainer" containerID="92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac" Jul 15 16:46:04.810906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.810630 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d" event={"ID":"c404be61-9668-4963-8e10-52aaa9ad578f","Type":"ContainerDied","Data":"6818cfa6ae3b9a70d9b97e2f50ebf63b02df8d4c224459e9ad3ca9657ff58770"} Jul 15 16:46:04.819181 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.819155 2566 scope.go:117] "RemoveContainer" containerID="92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac" Jul 15 16:46:04.819468 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:46:04.819447 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac\": container with ID starting with 92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac not found: ID does not exist" containerID="92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac" Jul 15 16:46:04.819572 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.819474 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac"} err="failed to get container status \"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac\": rpc error: code = NotFound desc = could not find container \"92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac\": container with ID starting with 92a1d53d7c99a7ab4713dec53c2f5b81aa9f12b10e73952d3b4c95c1b8c863ac not found: ID does not exist" Jul 15 16:46:04.840384 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.840358 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:46:04.850054 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.849884 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-8684c9ff49-gjc5d"] Jul 15 16:46:04.962145 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:04.962121 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:46:04.965046 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:46:04.965022 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9759ff72_8e63_4052_b85d_df18d84c2eff.slice/crio-30d485c4df60a1d45e0b63102f64b725e85e63df4e51d3240ce97fd1835f0bbf WatchSource:0}: Error finding container 30d485c4df60a1d45e0b63102f64b725e85e63df4e51d3240ce97fd1835f0bbf: Status 404 returned error can't find the container with id 30d485c4df60a1d45e0b63102f64b725e85e63df4e51d3240ce97fd1835f0bbf Jul 15 16:46:05.818500 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:05.818461 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" event={"ID":"9759ff72-8e63-4052-b85d-df18d84c2eff","Type":"ContainerStarted","Data":"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189"} Jul 15 16:46:05.818901 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:05.818521 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" event={"ID":"9759ff72-8e63-4052-b85d-df18d84c2eff","Type":"ContainerStarted","Data":"30d485c4df60a1d45e0b63102f64b725e85e63df4e51d3240ce97fd1835f0bbf"} Jul 15 16:46:05.818901 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:05.818546 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:05.841604 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:05.841556 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" podStartSLOduration=1.84153996 podStartE2EDuration="1.84153996s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:46:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:46:05.839519936 +0000 UTC m=+535.855891286" watchObservedRunningTime="2026-07-15 16:46:05.84153996 +0000 UTC m=+535.857911310" Jul 15 16:46:06.524078 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:06.524043 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c404be61-9668-4963-8e10-52aaa9ad578f" path="/var/lib/kubelet/pods/c404be61-9668-4963-8e10-52aaa9ad578f/volumes" Jul 15 16:46:13.353302 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:13.353245 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:13.353698 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:13.353347 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:13.358070 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:13.358046 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:13.862797 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:13.862770 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-6bc974cd7d-h8kkr" Jul 15 16:46:14.108059 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:14.108024 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:14.204954 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:14.204881 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:14.305603 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:14.305573 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:16.826571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:16.826544 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:46:20.649479 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:20.649444 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:20.762215 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:20.762180 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:33.282082 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.282045 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:33.372826 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.372791 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:33.444723 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.444693 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:33.730399 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.730358 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:46:33.733235 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.733214 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.735872 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.735854 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\"" Jul 15 16:46:33.745375 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.745346 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:46:33.759454 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.759420 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:46:33.852010 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.851980 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6221f688-e20c-477d-a72c-b0c014dd0fcf-config-file\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.852189 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.852016 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbk8v\" (UniqueName: \"kubernetes.io/projected/6221f688-e20c-477d-a72c-b0c014dd0fcf-kube-api-access-wbk8v\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.953211 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.953172 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6221f688-e20c-477d-a72c-b0c014dd0fcf-config-file\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.953415 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.953243 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wbk8v\" (UniqueName: \"kubernetes.io/projected/6221f688-e20c-477d-a72c-b0c014dd0fcf-kube-api-access-wbk8v\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.953831 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.953808 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/6221f688-e20c-477d-a72c-b0c014dd0fcf-config-file\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:33.964627 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:33.964600 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbk8v\" (UniqueName: \"kubernetes.io/projected/6221f688-e20c-477d-a72c-b0c014dd0fcf-kube-api-access-wbk8v\") pod \"limitador-limitador-54d6d988d5-f67jm\" (UID: \"6221f688-e20c-477d-a72c-b0c014dd0fcf\") " pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:34.052216 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:34.052139 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:34.104295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:34.104246 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:34.199838 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:34.199806 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:34.206140 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:34.206095 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:46:34.208569 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:46:34.208546 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6221f688_e20c_477d_a72c_b0c014dd0fcf.slice/crio-d08745089d04c578547683e88faae775c6c9471cab690be1c25df88c2e29c920 WatchSource:0}: Error finding container d08745089d04c578547683e88faae775c6c9471cab690be1c25df88c2e29c920: Status 404 returned error can't find the container with id d08745089d04c578547683e88faae775c6c9471cab690be1c25df88c2e29c920 Jul 15 16:46:34.969076 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:34.968994 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" event={"ID":"6221f688-e20c-477d-a72c-b0c014dd0fcf","Type":"ContainerStarted","Data":"d08745089d04c578547683e88faae775c6c9471cab690be1c25df88c2e29c920"} Jul 15 16:46:36.982602 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:36.982565 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" event={"ID":"6221f688-e20c-477d-a72c-b0c014dd0fcf","Type":"ContainerStarted","Data":"08a56fd70dc7bb3dd57e13af8d2d1ea3db72e2c6edbcb7eff3dfa9143c345a84"} Jul 15 16:46:36.982942 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:36.982707 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:37.004141 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:37.004096 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" podStartSLOduration=1.42164121 podStartE2EDuration="4.004083716s" totalImagesPullingTime="2.582442506s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:46:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:46:37.002112556 +0000 UTC m=+567.018483910" watchObservedRunningTime="2026-07-15 16:46:37.004083716 +0000 UTC m=+567.020455066" Jul 15 16:46:47.990382 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:47.990353 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-54d6d988d5-f67jm" Jul 15 16:46:51.738201 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:51.738162 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:51.840946 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:51.840917 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:46:52.248800 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:46:52.248770 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:04.786523 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:04.786490 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:05.054560 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:05.054482 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:10.390013 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:10.389971 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:47:10.391718 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:10.391696 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:47:12.808449 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:12.808416 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:12.903749 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:12.903714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:12.984844 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:12.984815 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:13.214390 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:13.214359 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:16.598757 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:16.598709 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:17.485906 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:17.485880 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:39.341424 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:39.341320 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:50.767411 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:50.767314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:50.968368 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:50.968314 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:51.360012 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:51.359982 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:51.931404 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:51.931371 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:52.036474 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.036437 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:47:52.081680 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.081651 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:47:52.121354 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.121306 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-69f5w"] Jul 15 16:47:52.125157 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.125137 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.127970 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.127948 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"payload-processing-dockercfg-9n7nq\"" Jul 15 16:47:52.128097 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.128011 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"payload-processing-plugins\"" Jul 15 16:47:52.136336 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.136295 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-69f5w"] Jul 15 16:47:52.206396 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.206312 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.206536 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.206413 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8mnx\" (UniqueName: \"kubernetes.io/projected/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-kube-api-access-t8mnx\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.307510 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.307479 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.307665 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.307523 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t8mnx\" (UniqueName: \"kubernetes.io/projected/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-kube-api-access-t8mnx\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.308140 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.308121 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"plugins-config-volume\" (UniqueName: \"kubernetes.io/configmap/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-plugins-config-volume\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.318750 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.318723 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8mnx\" (UniqueName: \"kubernetes.io/projected/4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21-kube-api-access-t8mnx\") pod \"payload-pre-processing-74584bb9f8-69f5w\" (UID: \"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21\") " pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.438221 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.438188 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:52.578185 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:52.578164 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/payload-pre-processing-74584bb9f8-69f5w"] Jul 15 16:47:52.580067 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:47:52.580039 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4fd038ea_f0a3_44e3_bb1f_e09e5f7bdf21.slice/crio-6ae18298813f7f83bece2d5a35b324cbf052f2e09cf67554fc68e567db0510b4 WatchSource:0}: Error finding container 6ae18298813f7f83bece2d5a35b324cbf052f2e09cf67554fc68e567db0510b4: Status 404 returned error can't find the container with id 6ae18298813f7f83bece2d5a35b324cbf052f2e09cf67554fc68e567db0510b4 Jul 15 16:47:53.363699 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:53.363667 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" event={"ID":"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21","Type":"ContainerStarted","Data":"6ae18298813f7f83bece2d5a35b324cbf052f2e09cf67554fc68e567db0510b4"} Jul 15 16:47:54.370426 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:54.370383 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" event={"ID":"4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21","Type":"ContainerStarted","Data":"6f86e6bbf7c0452da1bb28da41decb9f3067373df9d3e3e57453612d4a1a0f4a"} Jul 15 16:47:54.370881 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:54.370510 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:47:54.388562 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:47:54.388516 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" podStartSLOduration=0.779709452 podStartE2EDuration="2.388500041s" totalImagesPullingTime="1.608790589s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:47:52 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:47:54.388043152 +0000 UTC m=+644.404414503" watchObservedRunningTime="2026-07-15 16:47:54.388500041 +0000 UTC m=+644.404871393" Jul 15 16:48:02.757000 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:02.756960 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:02.848161 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:02.848133 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:05.377738 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:05.377708 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/payload-pre-processing-74584bb9f8-69f5w" Jul 15 16:48:19.284647 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:19.284609 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:19.757995 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:19.757963 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:48:29.246529 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.246498 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k"] Jul 15 16:48:29.252787 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.252762 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.256640 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.256612 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jul 15 16:48:29.256640 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.256633 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jul 15 16:48:29.256840 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.256611 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mpv6f\"" Jul 15 16:48:29.256840 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.256611 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jul 15 16:48:29.261363 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.261337 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k"] Jul 15 16:48:29.278222 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.278196 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:29.311153 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311126 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t7xzq\" (UniqueName: \"kubernetes.io/projected/d9325cec-0907-40cd-906f-dcef0c23adda-kube-api-access-t7xzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.311282 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311173 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.311282 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311239 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.311282 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311254 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.311282 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311269 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9325cec-0907-40cd-906f-dcef0c23adda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.311526 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.311428 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412669 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412639 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412677 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412700 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9325cec-0907-40cd-906f-dcef0c23adda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412770 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412804 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-t7xzq\" (UniqueName: \"kubernetes.io/projected/d9325cec-0907-40cd-906f-dcef0c23adda-kube-api-access-t7xzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.412851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.412841 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.413126 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.413085 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-home\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.413186 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.413137 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-model-cache\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.413186 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.413157 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-tmp-dir\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.415142 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.415122 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/d9325cec-0907-40cd-906f-dcef0c23adda-dshm\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.415315 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.415299 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/d9325cec-0907-40cd-906f-dcef0c23adda-tls-certs\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.420607 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.420590 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-t7xzq\" (UniqueName: \"kubernetes.io/projected/d9325cec-0907-40cd-906f-dcef0c23adda-kube-api-access-t7xzq\") pod \"e2e-distinct-simulated-kserve-59975d69f9-5kr2k\" (UID: \"d9325cec-0907-40cd-906f-dcef0c23adda\") " pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.564764 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.564683 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:29.707498 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.707474 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k"] Jul 15 16:48:29.709719 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:48:29.709679 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd9325cec_0907_40cd_906f_dcef0c23adda.slice/crio-02733afc42b238a54a1fbcfea29195896b6be2cac1ec5c9395b05a1b8a47b999 WatchSource:0}: Error finding container 02733afc42b238a54a1fbcfea29195896b6be2cac1ec5c9395b05a1b8a47b999: Status 404 returned error can't find the container with id 02733afc42b238a54a1fbcfea29195896b6be2cac1ec5c9395b05a1b8a47b999 Jul 15 16:48:29.711004 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.710987 2566 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 15 16:48:29.850430 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:29.850357 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:48:30.555884 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:30.555844 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" event={"ID":"d9325cec-0907-40cd-906f-dcef0c23adda","Type":"ContainerStarted","Data":"02733afc42b238a54a1fbcfea29195896b6be2cac1ec5c9395b05a1b8a47b999"} Jul 15 16:48:31.562447 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:31.562414 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" event={"ID":"d9325cec-0907-40cd-906f-dcef0c23adda","Type":"ContainerStarted","Data":"97bf8854160ff7d2e98cbce01c15c2bde9ddbedcf53cb0a1a9f144195b3e1050"} Jul 15 16:48:31.581292 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:31.581245 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" podStartSLOduration=1.5591967740000001 podStartE2EDuration="2.581230802s" totalImagesPullingTime="1.022034028s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:48:29 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:48:31.579881286 +0000 UTC m=+681.596252664" watchObservedRunningTime="2026-07-15 16:48:31.581230802 +0000 UTC m=+681.597602152" Jul 15 16:48:33.399797 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:33.399755 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:34.247549 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:34.247513 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:48:39.565212 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:39.565177 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:39.565645 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:39.565235 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:39.577701 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:39.577681 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:39.615086 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:39.615063 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-59975d69f9-5kr2k" Jul 15 16:48:41.045189 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:41.045155 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:42.455440 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:42.455406 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:48:50.638442 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.638408 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr"] Jul 15 16:48:50.641920 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.641903 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.644445 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.644427 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"facebook-opt-125m-simulated-kserve-self-signed-certs\"" Jul 15 16:48:50.652303 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.652276 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr"] Jul 15 16:48:50.668766 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.668736 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:48:50.698889 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.698864 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.699011 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.698893 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.699011 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.698918 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cjrgw\" (UniqueName: \"kubernetes.io/projected/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-kube-api-access-cjrgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.699011 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.698993 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.699117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.699055 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.699151 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.699119 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800093 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800014 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800093 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800061 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800110 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800124 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800144 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-cjrgw\" (UniqueName: \"kubernetes.io/projected/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-kube-api-access-cjrgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800295 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800179 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800684 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800662 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-model-cache\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800767 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800665 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-home\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.800767 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.800753 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tmp-dir\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.802485 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.802466 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-dshm\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.802657 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.802641 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-tls-certs\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.808372 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.808348 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-cjrgw\" (UniqueName: \"kubernetes.io/projected/eb80e93f-7025-44a7-bf90-6d4fe3d9751a-kube-api-access-cjrgw\") pod \"facebook-opt-125m-simulated-kserve-74b68447df-rtczr\" (UID: \"eb80e93f-7025-44a7-bf90-6d4fe3d9751a\") " pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:50.953397 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:50.953363 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:48:51.086809 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:51.086782 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr"] Jul 15 16:48:51.088437 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:48:51.088409 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podeb80e93f_7025_44a7_bf90_6d4fe3d9751a.slice/crio-88cdae09f665db1f8717c500554561b63be1f7e5e17ee0a65c0dd6259439460d WatchSource:0}: Error finding container 88cdae09f665db1f8717c500554561b63be1f7e5e17ee0a65c0dd6259439460d: Status 404 returned error can't find the container with id 88cdae09f665db1f8717c500554561b63be1f7e5e17ee0a65c0dd6259439460d Jul 15 16:48:51.664571 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:51.664540 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" event={"ID":"eb80e93f-7025-44a7-bf90-6d4fe3d9751a","Type":"ContainerStarted","Data":"2caf95c355cefabd696893180f6b19f6248ccb35e0908867a7ee9531a3016bba"} Jul 15 16:48:51.664969 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:51.664590 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" event={"ID":"eb80e93f-7025-44a7-bf90-6d4fe3d9751a","Type":"ContainerStarted","Data":"88cdae09f665db1f8717c500554561b63be1f7e5e17ee0a65c0dd6259439460d"} Jul 15 16:48:51.682951 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:51.682903 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" podStartSLOduration=1.511710897 podStartE2EDuration="1.682888027s" totalImagesPullingTime="171.17713ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:48:50 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:48:51.681974794 +0000 UTC m=+701.698346145" watchObservedRunningTime="2026-07-15 16:48:51.682888027 +0000 UTC m=+701.699259377" Jul 15 16:48:52.750361 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:52.750313 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:48:54.386762 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:48:54.386720 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:49:00.954383 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:00.954341 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:49:00.954793 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:00.954408 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:49:00.966839 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:00.966813 2566 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:49:01.550257 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:01.550214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:49:01.734321 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:01.734291 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/facebook-opt-125m-simulated-kserve-74b68447df-rtczr" Jul 15 16:49:25.202266 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:25.202236 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:49:58.838400 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:49:58.838318 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:10.948935 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:10.948899 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:17.778154 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:17.778093 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:17.879484 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:17.879443 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:18.721924 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:18.721877 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:18.846622 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:18.846576 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:37.562225 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:37.562149 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:37.618791 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:37.618764 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:55.166918 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:55.166879 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:50:56.242202 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:56.242171 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:50:56.880408 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:50:56.880370 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:51:05.056619 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:05.056586 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:51:15.646831 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:15.646792 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:51:24.848704 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:24.848670 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:51:35.649931 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:35.649899 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:51:38.959312 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:38.959278 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:51:40.016961 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:51:40.016927 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:01.655620 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:01.655588 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:52:10.432389 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:10.432301 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:52:10.432821 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:10.432477 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:52:16.781368 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:16.781308 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:52:20.870178 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:20.870145 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:21.631554 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:21.631516 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:21.732497 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:21.732467 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:26.985546 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:26.985514 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:27.091806 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:27.091751 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:27.186005 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:27.185974 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:27.341431 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:27.341352 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:33.447786 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:33.447748 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:33.533146 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:33.533111 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:47.495711 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:47.495675 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:47.595268 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:47.595237 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:52:50.353041 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:50.353008 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:52:59.968432 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:52:59.968402 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:09.257522 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:09.257482 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:17.748136 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:17.748099 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:26.463610 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:26.463577 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:28.310690 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:28.310656 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:53:28.408117 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:28.408086 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:53:35.953153 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:35.953072 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:47.958823 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:47.958777 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:53:57.867852 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:53:57.867816 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:54:20.667477 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:54:20.667444 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:54:29.726214 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:54:29.726180 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:54:32.681393 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:54:32.681358 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:54:52.952550 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:54:52.952515 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:55:02.955729 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:55:02.955647 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:55:20.667598 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:55:20.667568 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:55:29.369107 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:55:29.369074 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:55:39.465851 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:55:39.465814 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:55:55.257196 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:55:55.257162 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:56:00.565189 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:00.565152 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 16:56:25.095915 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:25.095882 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:56:25.096387 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:25.096069 2566 kuberuntime_container.go:873] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" podUID="9759ff72-8e63-4052-b85d-df18d84c2eff" containerName="manager" containerID="cri-o://f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189" gracePeriod=10 Jul 15 16:56:28.252752 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.252728 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:56:28.303862 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.303829 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume\") pod \"9759ff72-8e63-4052-b85d-df18d84c2eff\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " Jul 15 16:56:28.304016 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.303886 2566 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9v8x9\" (UniqueName: \"kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9\") pod \"9759ff72-8e63-4052-b85d-df18d84c2eff\" (UID: \"9759ff72-8e63-4052-b85d-df18d84c2eff\") " Jul 15 16:56:28.304351 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.304303 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "9759ff72-8e63-4052-b85d-df18d84c2eff" (UID: "9759ff72-8e63-4052-b85d-df18d84c2eff"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 15 16:56:28.305960 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.305940 2566 operation_generator.go:782] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9" (OuterVolumeSpecName: "kube-api-access-9v8x9") pod "9759ff72-8e63-4052-b85d-df18d84c2eff" (UID: "9759ff72-8e63-4052-b85d-df18d84c2eff"). InnerVolumeSpecName "kube-api-access-9v8x9". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 15 16:56:28.405677 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.405603 2566 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/9759ff72-8e63-4052-b85d-df18d84c2eff-extensions-socket-volume\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:56:28.405677 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:28.405629 2566 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9v8x9\" (UniqueName: \"kubernetes.io/projected/9759ff72-8e63-4052-b85d-df18d84c2eff-kube-api-access-9v8x9\") on node \"ip-10-0-138-140.ec2.internal\" DevicePath \"\"" Jul 15 16:56:29.023382 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.023350 2566 generic.go:356] "Generic (PLEG): container finished" podID="9759ff72-8e63-4052-b85d-df18d84c2eff" containerID="f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189" exitCode=0 Jul 15 16:56:29.023559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.023395 2566 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" Jul 15 16:56:29.023559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.023424 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" event={"ID":"9759ff72-8e63-4052-b85d-df18d84c2eff","Type":"ContainerDied","Data":"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189"} Jul 15 16:56:29.023559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.023482 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm" event={"ID":"9759ff72-8e63-4052-b85d-df18d84c2eff","Type":"ContainerDied","Data":"30d485c4df60a1d45e0b63102f64b725e85e63df4e51d3240ce97fd1835f0bbf"} Jul 15 16:56:29.023559 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.023502 2566 scope.go:117] "RemoveContainer" containerID="f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189" Jul 15 16:56:29.033224 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.033208 2566 scope.go:117] "RemoveContainer" containerID="f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189" Jul 15 16:56:29.033510 ip-10-0-138-140 kubenswrapper[2566]: E0715 16:56:29.033491 2566 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189\": container with ID starting with f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189 not found: ID does not exist" containerID="f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189" Jul 15 16:56:29.033578 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.033518 2566 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189"} err="failed to get container status \"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189\": rpc error: code = NotFound desc = could not find container \"f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189\": container with ID starting with f4b38859b1b81997ed3fc2f15bc76b9f4198f51867fc30dd0a1b1b741b49f189 not found: ID does not exist" Jul 15 16:56:29.046289 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.046268 2566 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:56:29.051665 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:29.051644 2566 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-g8lrm"] Jul 15 16:56:30.528817 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:56:30.528775 2566 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9759ff72-8e63-4052-b85d-df18d84c2eff" path="/var/lib/kubelet/pods/9759ff72-8e63-4052-b85d-df18d84c2eff/volumes" Jul 15 16:57:10.464212 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:10.464185 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:57:10.466491 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:10.466471 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 16:57:26.254471 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.254435 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x"] Jul 15 16:57:26.256921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.254848 2566 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9759ff72-8e63-4052-b85d-df18d84c2eff" containerName="manager" Jul 15 16:57:26.256921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.254858 2566 state_mem.go:107] "Deleted CPUSet assignment" podUID="9759ff72-8e63-4052-b85d-df18d84c2eff" containerName="manager" Jul 15 16:57:26.256921 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.254938 2566 memory_manager.go:368] "RemoveStaleState removing state" podUID="9759ff72-8e63-4052-b85d-df18d84c2eff" containerName="manager" Jul 15 16:57:26.257917 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.257902 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.261412 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.261387 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-4mv6t\"" Jul 15 16:57:26.271662 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.271640 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x"] Jul 15 16:57:26.278428 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.278403 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wmv66\" (UniqueName: \"kubernetes.io/projected/8e35c6ed-3315-4907-85dd-a918ac261808-kube-api-access-wmv66\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.278535 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.278494 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e35c6ed-3315-4907-85dd-a918ac261808-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.379875 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.379846 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wmv66\" (UniqueName: \"kubernetes.io/projected/8e35c6ed-3315-4907-85dd-a918ac261808-kube-api-access-wmv66\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.380024 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.379901 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e35c6ed-3315-4907-85dd-a918ac261808-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.380213 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.380199 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/8e35c6ed-3315-4907-85dd-a918ac261808-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.392895 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.392870 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-wmv66\" (UniqueName: \"kubernetes.io/projected/8e35c6ed-3315-4907-85dd-a918ac261808-kube-api-access-wmv66\") pod \"kuadrant-operator-controller-manager-86878c769f-fl56x\" (UID: \"8e35c6ed-3315-4907-85dd-a918ac261808\") " pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.568165 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.568091 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:26.705264 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:26.705233 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x"] Jul 15 16:57:26.707419 ip-10-0-138-140 kubenswrapper[2566]: W0715 16:57:26.707392 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e35c6ed_3315_4907_85dd_a918ac261808.slice/crio-6c1e2d0278197f8f25ed05a5ebbd3669083c22489f89af9b3da5c005e19174dc WatchSource:0}: Error finding container 6c1e2d0278197f8f25ed05a5ebbd3669083c22489f89af9b3da5c005e19174dc: Status 404 returned error can't find the container with id 6c1e2d0278197f8f25ed05a5ebbd3669083c22489f89af9b3da5c005e19174dc Jul 15 16:57:27.323304 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:27.323268 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" event={"ID":"8e35c6ed-3315-4907-85dd-a918ac261808","Type":"ContainerStarted","Data":"fa83d44cd9a0b27ab6fe0e272dd2dcab2c4b2b6bada3c067fc5c0ce2950deb8d"} Jul 15 16:57:27.323720 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:27.323385 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:27.323720 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:27.323407 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" event={"ID":"8e35c6ed-3315-4907-85dd-a918ac261808","Type":"ContainerStarted","Data":"6c1e2d0278197f8f25ed05a5ebbd3669083c22489f89af9b3da5c005e19174dc"} Jul 15 16:57:27.355392 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:27.355341 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" podStartSLOduration=1.355311148 podStartE2EDuration="1.355311148s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-15 16:57:26 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 16:57:27.351567537 +0000 UTC m=+1217.367938968" watchObservedRunningTime="2026-07-15 16:57:27.355311148 +0000 UTC m=+1217.371682498" Jul 15 16:57:38.331489 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:38.331449 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-86878c769f-fl56x" Jul 15 16:57:46.954786 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:46.954742 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:57:57.160795 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:57:57.160752 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:58:07.760840 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:58:07.760761 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:58:18.652208 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:58:18.652174 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:58:30.147880 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:58:30.147837 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:58:40.864219 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:58:40.864182 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:58:50.150738 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:58:50.150702 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:59:05.963570 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:59:05.963529 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:59:15.970541 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:59:15.970507 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:59:24.553841 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:59:24.553808 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:59:37.471256 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:59:37.471182 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 16:59:45.769103 ip-10-0-138-140 kubenswrapper[2566]: I0715 16:59:45.769064 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:00:27.451249 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:00:27.451214 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:00:36.550384 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:00:36.550349 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:00:47.551119 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:00:47.551084 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:02:10.496131 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:02:10.496100 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 17:02:10.500242 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:02:10.500222 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-z2n2r_617c55d3-8cef-4dc0-8b1a-9e81b4ae8bb3/ovn-acl-logging/0.log" Jul 15 17:02:14.220603 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:02:14.220562 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 17:02:54.352034 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:02:54.351958 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:03:06.197854 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:06.197819 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:03:17.555061 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:17.555024 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:03:30.451811 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:30.451779 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-54d6d988d5-f67jm"] Jul 15 17:03:38.522310 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:38.522277 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fnj4p_b57c744b-2dc0-439a-8fff-056e5c921e4e/manager/0.log" Jul 15 17:03:38.645172 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:38.645147 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ncsmh_141cee3a-4464-4281-9b50-56f4a43d94aa/manager/0.log" Jul 15 17:03:38.897646 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:38.897559 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gp54s_7225bdf8-a4c5-4b07-b948-10077dbb206d/server/0.log" Jul 15 17:03:39.303068 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:39.303032 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f48c5d8-4nl2s_e2477bfc-22c7-4f48-85f0-3b13443b5598/copy-manifests/0.log" Jul 15 17:03:39.320817 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:39.320785 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f48c5d8-4nl2s_e2477bfc-22c7-4f48-85f0-3b13443b5598/manager/0.log" Jul 15 17:03:41.732602 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:41.732568 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wtj99_fa01296e-388b-4ed4-8c72-898216c99193/manager/0.log" Jul 15 17:03:41.848872 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:41.848839 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gtpcr_0e9d901b-d3a2-447a-beb1-c742ed401f4e/manager/0.log" Jul 15 17:03:41.972215 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:41.972189 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sd6l4_f212bb1e-f489-4b2e-8deb-446a26f9d8a4/kuadrant-console-plugin/0.log" Jul 15 17:03:42.105881 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:42.105810 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hfb8j_57ec28fe-92d0-483f-bbbe-912ec1464213/registry-server/0.log" Jul 15 17:03:42.261856 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:42.261807 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fl56x_8e35c6ed-3315-4907-85dd-a918ac261808/manager/0.log" Jul 15 17:03:42.383108 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:42.383026 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f67jm_6221f688-e20c-477d-a72c-b0c014dd0fcf/limitador/0.log" Jul 15 17:03:42.508758 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:42.508713 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h4g4r_8cf0ff67-f438-4b11-aee3-a4c1d0f61c71/manager/0.log" Jul 15 17:03:43.152218 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:43.152182 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df86594bb-b4dvs_806e589e-a2cc-4f84-9bf8-b9bca4de08f3/kube-auth-proxy/0.log" Jul 15 17:03:43.552584 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:43.552553 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-69f5w_4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21/payload-pre-processing/0.log" Jul 15 17:03:43.810624 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:43.810547 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-698f89cbfd-mss4k_bf3eae5b-98c4-4248-a052-44f3518fff74/router/0.log" Jul 15 17:03:44.295376 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:44.295346 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-59975d69f9-5kr2k_d9325cec-0907-40cd-906f-dcef0c23adda/main/0.log" Jul 15 17:03:44.638560 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:44.638456 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_facebook-opt-125m-simulated-kserve-74b68447df-rtczr_eb80e93f-7025-44a7-bf90-6d4fe3d9751a/main/0.log" Jul 15 17:03:51.244376 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:51.244315 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:53.663576 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:53.663543 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:54.050497 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:54.050469 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-gv5w9_e38e6b54-e6dd-46e2-820a-347c749cb0b9/global-pull-secret-syncer/0.log" Jul 15 17:03:54.145299 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:54.145270 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" Jul 15 17:03:54.254920 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:54.254892 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-140.ec2.internal_a8eeed7395da0aa41ef27a523d3170f4/haproxy/0.log" Jul 15 17:03:57.736751 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:57.736714 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:57.798193 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:57.798161 2566 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-bjvfb_6396047c-2712-4f87-b933-b00ad41b5d9c/konnectivity-agent/0.log" limit=1024 Jul 15 17:03:59.745860 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.745829 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-75cc8d5cb7-wtj99_fa01296e-388b-4ed4-8c72-898216c99193/manager/0.log" Jul 15 17:03:59.772962 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.772938 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_dns-operator-controller-manager-864dd78c4c-gtpcr_0e9d901b-d3a2-447a-beb1-c742ed401f4e/manager/0.log" Jul 15 17:03:59.806217 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.806184 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-66b5774985-sd6l4_f212bb1e-f489-4b2e-8deb-446a26f9d8a4/kuadrant-console-plugin/0.log" Jul 15 17:03:59.845811 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.845786 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-hfb8j_57ec28fe-92d0-483f-bbbe-912ec1464213/registry-server/0.log" Jul 15 17:03:59.919074 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.919045 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-controller-manager-86878c769f-fl56x_8e35c6ed-3315-4907-85dd-a918ac261808/manager/0.log" Jul 15 17:03:59.948006 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:03:59.947984 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-limitador-54d6d988d5-f67jm_6221f688-e20c-477d-a72c-b0c014dd0fcf/limitador/0.log" Jul 15 17:04:00.015822 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:00.015719 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_limitador-operator-controller-manager-859d7b78bd-h4g4r_8cf0ff67-f438-4b11-aee3-a4c1d0f61c71/manager/0.log" Jul 15 17:04:01.703803 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.703779 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/alertmanager/0.log" Jul 15 17:04:01.725236 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.725212 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/config-reloader/0.log" Jul 15 17:04:01.755021 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.754998 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/kube-rbac-proxy-web/0.log" Jul 15 17:04:01.782901 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.782873 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/kube-rbac-proxy/0.log" Jul 15 17:04:01.808781 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.808757 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/kube-rbac-proxy-metric/0.log" Jul 15 17:04:01.834987 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.834969 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/prom-label-proxy/0.log" Jul 15 17:04:01.858611 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.858585 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_aaabce8b-3db3-4eb0-acac-701e35db4d60/init-config-reloader/0.log" Jul 15 17:04:01.907183 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:01.907128 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-5f7f494dd4-gfspr_5bcb7a32-be70-49c9-a0eb-0131ccc90a61/cluster-monitoring-operator/0.log" Jul 15 17:04:02.057864 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.057833 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5f68df7f58-bgshv_7dbcd7d3-bb9a-4206-a788-d01e559e0ad8/monitoring-plugin/0.log" Jul 15 17:04:02.254814 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.254785 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxnqd_f1e906a7-ea5d-4393-9573-5461c28b2681/node-exporter/0.log" Jul 15 17:04:02.278041 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.278018 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxnqd_f1e906a7-ea5d-4393-9573-5461c28b2681/kube-rbac-proxy/0.log" Jul 15 17:04:02.300591 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.300573 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cxnqd_f1e906a7-ea5d-4393-9573-5461c28b2681/init-textfile/0.log" Jul 15 17:04:02.330104 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.330079 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcl6d_f6766722-4419-4487-82cd-f231339815a1/kube-rbac-proxy-main/0.log" Jul 15 17:04:02.354545 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.354528 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcl6d_f6766722-4419-4487-82cd-f231339815a1/kube-rbac-proxy-self/0.log" Jul 15 17:04:02.382673 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.382654 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-84bd68b896-mcl6d_f6766722-4419-4487-82cd-f231339815a1/openshift-state-metrics/0.log" Jul 15 17:04:02.648906 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.648836 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-admission-webhook-77d98bc786-qrnkp_96b31b8a-9e99-47e0-9acf-dbec747d8d20/prometheus-operator-admission-webhook/0.log" Jul 15 17:04:02.679758 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.679730 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86656545ff-lwz8d_d46b1191-0831-4629-9444-b7c326519fb9/telemeter-client/0.log" Jul 15 17:04:02.702824 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.702795 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86656545ff-lwz8d_d46b1191-0831-4629-9444-b7c326519fb9/reload/0.log" Jul 15 17:04:02.723732 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.723713 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-86656545ff-lwz8d_d46b1191-0831-4629-9444-b7c326519fb9/kube-rbac-proxy/0.log" Jul 15 17:04:02.770172 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.770146 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/thanos-query/0.log" Jul 15 17:04:02.794622 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.794600 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/kube-rbac-proxy-web/0.log" Jul 15 17:04:02.819219 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.819199 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/kube-rbac-proxy/0.log" Jul 15 17:04:02.842146 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.842120 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/prom-label-proxy/0.log" Jul 15 17:04:02.864870 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.864843 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/kube-rbac-proxy-rules/0.log" Jul 15 17:04:02.888556 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:02.888535 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-786b8476f7-ww2zf_da5a4a1c-95b7-4a4b-a90b-136d4b12e252/kube-rbac-proxy-metrics/0.log" Jul 15 17:04:04.188835 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.188803 2566 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb"] Jul 15 17:04:04.192760 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.192714 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.196108 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.196083 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2m4bp\"/\"kube-root-ca.crt\"" Jul 15 17:04:04.196252 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.196138 2566 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-2m4bp\"/\"openshift-service-ca.crt\"" Jul 15 17:04:04.197219 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.197201 2566 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-2m4bp\"/\"default-dockercfg-r28gz\"" Jul 15 17:04:04.202345 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.202304 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb"] Jul 15 17:04:04.339196 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.339164 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ldh9j\" (UniqueName: \"kubernetes.io/projected/a954c1ef-9ac8-4168-99a6-a18b5087f956-kube-api-access-ldh9j\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.339415 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.339204 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-sys\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.339415 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.339226 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-proc\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.339415 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.339315 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-lib-modules\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.339415 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.339399 2566 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-podres\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440598 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440512 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-ldh9j\" (UniqueName: \"kubernetes.io/projected/a954c1ef-9ac8-4168-99a6-a18b5087f956-kube-api-access-ldh9j\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440598 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440553 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-sys\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440787 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440618 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-sys\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440787 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440641 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-proc\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440787 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440680 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-lib-modules\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440787 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440710 2566 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-podres\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440787 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440767 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-proc\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440974 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440813 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-lib-modules\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.440974 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.440823 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/a954c1ef-9ac8-4168-99a6-a18b5087f956-podres\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.448868 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.448840 2566 operation_generator.go:616] "MountVolume.SetUp succeeded for volume \"kube-api-access-ldh9j\" (UniqueName: \"kubernetes.io/projected/a954c1ef-9ac8-4168-99a6-a18b5087f956-kube-api-access-ldh9j\") pod \"perf-node-gather-daemonset-fwnnb\" (UID: \"a954c1ef-9ac8-4168-99a6-a18b5087f956\") " pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.503934 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.503901 2566 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:04.637466 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:04.637440 2566 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb"] Jul 15 17:04:04.639477 ip-10-0-138-140 kubenswrapper[2566]: W0715 17:04:04.639448 2566 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-poda954c1ef_9ac8_4168_99a6_a18b5087f956.slice/crio-112b6da71b33ad5b303564e8131cd63784192cf72f1764ff6835934f9bd3013f WatchSource:0}: Error finding container 112b6da71b33ad5b303564e8131cd63784192cf72f1764ff6835934f9bd3013f: Status 404 returned error can't find the container with id 112b6da71b33ad5b303564e8131cd63784192cf72f1764ff6835934f9bd3013f Jul 15 17:04:05.145577 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.145549 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-6bc974cd7d-h8kkr_7c98d1cf-42e2-469f-8996-2f676f49db24/console/0.log" Jul 15 17:04:05.381854 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.381818 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" event={"ID":"a954c1ef-9ac8-4168-99a6-a18b5087f956","Type":"ContainerStarted","Data":"4136e7aa10ad5881f1f782b374b3b54d789267fc03ab9a0bc26940edc60a3593"} Jul 15 17:04:05.382214 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.381884 2566 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" event={"ID":"a954c1ef-9ac8-4168-99a6-a18b5087f956","Type":"ContainerStarted","Data":"112b6da71b33ad5b303564e8131cd63784192cf72f1764ff6835934f9bd3013f"} Jul 15 17:04:05.382214 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.381914 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:05.399218 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.399135 2566 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" podStartSLOduration=1.3991197020000001 podStartE2EDuration="1.399119702s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-15 17:04:04 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-15 17:04:05.398965824 +0000 UTC m=+1615.415337175" watchObservedRunningTime="2026-07-15 17:04:05.399119702 +0000 UTC m=+1615.415491070" Jul 15 17:04:05.773080 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:05.773050 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-78495597f6-xsgj2_5b918283-8655-4d4c-a4bf-1fd1cf5bb107/volume-data-source-validator/0.log" Jul 15 17:04:06.863821 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:06.863794 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ml26l_280bfa80-5182-48de-8e1e-a4dae02247fd/dns/0.log" Jul 15 17:04:06.883278 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:06.883251 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-ml26l_280bfa80-5182-48de-8e1e-a4dae02247fd/kube-rbac-proxy/0.log" Jul 15 17:04:06.980978 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:06.980954 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-w4t9b_3cafe0e2-688f-4634-9fe6-a039e8afc21b/dns-node-resolver/0.log" Jul 15 17:04:07.555474 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:07.555441 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6955c964f4-5tqp7_91e76f29-ffd4-4dd8-be9d-bdafa83e7dec/registry/0.log" Jul 15 17:04:07.565202 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:07.565177 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-6955c964f4-5tqp7_91e76f29-ffd4-4dd8-be9d-bdafa83e7dec/registry/1.log" Jul 15 17:04:07.587471 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:07.587447 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-g6ldc_cc95bc2d-0c04-41dc-b889-b33014478432/node-ca/0.log" Jul 15 17:04:08.687878 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:08.687842 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df86594bb-b4dvs_806e589e-a2cc-4f84-9bf8-b9bca4de08f3/kube-auth-proxy/0.log" Jul 15 17:04:08.805056 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:08.805030 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-69f5w_4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21/payload-pre-processing/0.log" Jul 15 17:04:08.855125 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:08.855097 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-698f89cbfd-mss4k_bf3eae5b-98c4-4248-a052-44f3518fff74/router/0.log" Jul 15 17:04:09.496605 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:09.496583 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-m7z8z_d031ba50-1ba1-46c5-9ab7-28268c46b659/serve-healthcheck-canary/0.log" Jul 15 17:04:10.140006 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:10.139976 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lczwb_fafc9049-86af-4af5-b9ed-cd5a5ad2fc00/kube-rbac-proxy/0.log" Jul 15 17:04:10.171981 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:10.171955 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lczwb_fafc9049-86af-4af5-b9ed-cd5a5ad2fc00/exporter/0.log" Jul 15 17:04:10.205602 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:10.205576 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lczwb_fafc9049-86af-4af5-b9ed-cd5a5ad2fc00/extractor/0.log" Jul 15 17:04:11.396801 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:11.396772 2566 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-2m4bp/perf-node-gather-daemonset-fwnnb" Jul 15 17:04:12.592826 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:12.592796 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_kserve-controller-manager-d6f4b65b8-fnj4p_b57c744b-2dc0-439a-8fff-056e5c921e4e/manager/0.log" Jul 15 17:04:12.676942 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:12.676916 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_llmisvc-controller-manager-777c6bd7d5-ncsmh_141cee3a-4464-4281-9b50-56f4a43d94aa/manager/0.log" Jul 15 17:04:12.782773 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:12.782747 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_model-serving-api-54d89bd5fc-gp54s_7225bdf8-a4c5-4b07-b948-10077dbb206d/server/0.log" Jul 15 17:04:13.004785 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:13.004755 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f48c5d8-4nl2s_e2477bfc-22c7-4f48-85f0-3b13443b5598/manager/0.log" Jul 15 17:04:13.025577 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:13.025555 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-67f48c5d8-4nl2s_e2477bfc-22c7-4f48-85f0-3b13443b5598/copy-manifests/0.log" Jul 15 17:04:14.514625 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:14.514593 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df86594bb-b4dvs_806e589e-a2cc-4f84-9bf8-b9bca4de08f3/kube-auth-proxy/0.log" Jul 15 17:04:14.599027 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:14.599002 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-69f5w_4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21/payload-pre-processing/0.log" Jul 15 17:04:14.628554 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:14.628525 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-698f89cbfd-mss4k_bf3eae5b-98c4-4248-a052-44f3518fff74/router/0.log" Jul 15 17:04:15.762460 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:15.762427 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-7df86594bb-b4dvs_806e589e-a2cc-4f84-9bf8-b9bca4de08f3/kube-auth-proxy/0.log" Jul 15 17:04:15.847053 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:15.847020 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_payload-pre-processing-74584bb9f8-69f5w_4fd038ea-f0a3-44e3-bb1f-e09e5f7bdf21/payload-pre-processing/0.log" Jul 15 17:04:15.876644 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:15.876617 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-698f89cbfd-mss4k_bf3eae5b-98c4-4248-a052-44f3518fff74/router/0.log" Jul 15 17:04:19.294972 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:19.294946 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-mc887_3347e66c-467c-4342-9d86-15e835b219c9/migrator/0.log" Jul 15 17:04:19.318277 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:19.318247 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-7f59b54f6c-mc887_3347e66c-467c-4342-9d86-15e835b219c9/graceful-termination/0.log" Jul 15 17:04:21.019744 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.019715 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/kube-multus-additional-cni-plugins/0.log" Jul 15 17:04:21.042960 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.042932 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/egress-router-binary-copy/0.log" Jul 15 17:04:21.063438 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.063414 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/cni-plugins/0.log" Jul 15 17:04:21.087544 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.087523 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/bond-cni-plugin/0.log" Jul 15 17:04:21.110175 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.110153 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/routeoverride-cni/0.log" Jul 15 17:04:21.134833 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.134814 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/whereabouts-cni-bincopy/0.log" Jul 15 17:04:21.156442 ip-10-0-138-140 kubenswrapper[2566]: I0715 17:04:21.156425 2566 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-cm6xf_0a75565a-dc09-41c9-92a3-0f3bab87860f/whereabouts-cni/0.log"