Jul 01 16:56:02.036227 ip-10-0-138-241 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jul 01 16:56:02.036237 ip-10-0-138-241 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jul 01 16:56:02.036244 ip-10-0-138-241 systemd[1]: kubelet.service: Failed with result 'resources'. Jul 01 16:56:02.036538 ip-10-0-138-241 systemd[1]: Failed to start Kubernetes Kubelet. Jul 01 16:56:12.238817 ip-10-0-138-241 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jul 01 16:56:12.238839 ip-10-0-138-241 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 08d0db4ff7ed43c6b12fe31824648ded -- Jul 01 16:57:56.640167 ip-10-0-138-241 systemd[1]: Starting Kubernetes Kubelet... Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.045208 2580 server.go:213] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050293 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050310 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050314 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050317 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050320 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050323 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050326 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.110665 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050336 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050341 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050345 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050350 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050352 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050355 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050359 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050361 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050364 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050367 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050369 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050372 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050374 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050377 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050379 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050382 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050385 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050388 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050391 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050394 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.112526 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050397 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050400 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050402 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050405 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050408 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050410 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050413 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050415 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050418 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050421 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050423 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050426 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050429 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050438 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050441 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050443 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050445 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050448 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050451 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050453 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.114168 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050456 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050459 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050462 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050466 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050469 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050472 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050474 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050477 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050479 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050483 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050486 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050489 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050492 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050498 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050500 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050503 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050506 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050508 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050511 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.115310 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050513 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050516 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050518 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050521 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050523 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050528 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050530 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050539 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050541 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050544 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050546 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050549 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050551 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050554 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050556 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050559 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050562 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050564 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050567 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050569 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.116145 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050573 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050576 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050578 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050581 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050585 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050588 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050592 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050595 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050598 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050600 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050603 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050606 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.050608 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.053981 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.053990 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.053994 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.053997 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054000 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054002 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054005 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.117039 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054008 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054011 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054014 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054016 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054019 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054022 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054024 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054027 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054030 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054032 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054035 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054037 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054040 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054042 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054045 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054048 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054050 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054055 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054060 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054063 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.117927 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054065 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054068 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054070 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054073 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054076 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054079 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054081 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054084 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054087 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054089 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054092 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054094 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054097 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054100 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054102 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054106 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054108 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054111 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054113 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054116 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.118826 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054118 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.119491 ip-10-0-138-241 systemd[1]: Started Kubernetes Kubelet. Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054120 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054123 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054125 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054128 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054130 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054134 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054138 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054141 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054143 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054148 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054150 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054153 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054155 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054158 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054160 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054163 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054166 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054169 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.119828 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054173 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054176 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054179 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054182 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054185 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054187 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054190 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054193 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054195 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054198 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054200 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054203 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054205 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054208 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054210 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054213 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054215 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054219 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054222 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054226 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.120690 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054229 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054232 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054234 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054238 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054241 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054243 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054246 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054248 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054251 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054253 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054257 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054260 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054263 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054695 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054700 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054703 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054707 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054710 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054712 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054715 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.121796 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054718 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054721 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054723 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054726 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054728 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054731 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054734 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054736 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054739 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054741 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054744 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054746 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054749 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054752 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054755 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054759 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054762 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054764 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054767 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054769 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.122708 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054772 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054774 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054777 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054780 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054782 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054784 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054787 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054789 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054792 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054795 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054798 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054820 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054825 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054828 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054831 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054834 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054836 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054838 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054841 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054843 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.123998 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054846 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054848 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054851 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054853 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054856 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054859 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054861 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054865 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054868 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054871 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054874 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054876 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054880 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054882 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054885 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054888 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054890 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054893 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054895 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054897 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.125281 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054900 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054902 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054906 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054910 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054913 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054917 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054919 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054922 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054924 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054928 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054932 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054935 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054937 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054940 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054943 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054945 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054948 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054950 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054953 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.126272 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054958 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054961 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054964 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054966 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054969 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054971 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054974 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054976 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054979 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054981 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054984 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054986 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.054989 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055056 2580 flags.go:64] FLAG: --address="0.0.0.0" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055064 2580 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055072 2580 flags.go:64] FLAG: --anonymous-auth="true" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055076 2580 flags.go:64] FLAG: --application-metrics-count-limit="100" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055080 2580 flags.go:64] FLAG: --authentication-token-webhook="false" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055083 2580 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055088 2580 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055093 2580 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jul 01 16:57:57.127133 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055096 2580 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055099 2580 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055102 2580 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055105 2580 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055109 2580 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055112 2580 flags.go:64] FLAG: --cgroup-root="" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055114 2580 flags.go:64] FLAG: --cgroups-per-qos="true" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055117 2580 flags.go:64] FLAG: --client-ca-file="" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055120 2580 flags.go:64] FLAG: --cloud-provider="external" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055123 2580 flags.go:64] FLAG: --cluster-dns="[]" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055128 2580 flags.go:64] FLAG: --cluster-domain="" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055132 2580 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055135 2580 flags.go:64] FLAG: --config-dir="/etc/openshift/kubelet.conf.d" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055138 2580 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055142 2580 flags.go:64] FLAG: --container-log-max-files="5" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055146 2580 flags.go:64] FLAG: --container-log-max-size="10Mi" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055148 2580 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055152 2580 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055155 2580 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055158 2580 flags.go:64] FLAG: --contention-profiling="false" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055161 2580 flags.go:64] FLAG: --cpu-cfs-quota="true" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055164 2580 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055167 2580 flags.go:64] FLAG: --cpu-manager-policy="none" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055170 2580 flags.go:64] FLAG: --cpu-manager-policy-options="" Jul 01 16:57:57.128214 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055174 2580 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055177 2580 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055181 2580 flags.go:64] FLAG: --enable-debugging-handlers="true" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055184 2580 flags.go:64] FLAG: --enable-load-reader="false" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055187 2580 flags.go:64] FLAG: --enable-server="true" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055190 2580 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055195 2580 flags.go:64] FLAG: --event-burst="100" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055198 2580 flags.go:64] FLAG: --event-qps="50" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055201 2580 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055204 2580 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055207 2580 flags.go:64] FLAG: --eviction-hard="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055211 2580 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055214 2580 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055217 2580 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055220 2580 flags.go:64] FLAG: --eviction-soft="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055223 2580 flags.go:64] FLAG: --eviction-soft-grace-period="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055226 2580 flags.go:64] FLAG: --exit-on-lock-contention="false" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055229 2580 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055232 2580 flags.go:64] FLAG: --experimental-mounter-path="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055236 2580 flags.go:64] FLAG: --fail-cgroupv1="false" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055239 2580 flags.go:64] FLAG: --fail-swap-on="true" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055242 2580 flags.go:64] FLAG: --feature-gates="" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055245 2580 flags.go:64] FLAG: --file-check-frequency="20s" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055248 2580 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055251 2580 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jul 01 16:57:57.129299 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055254 2580 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055258 2580 flags.go:64] FLAG: --healthz-port="10248" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055261 2580 flags.go:64] FLAG: --help="false" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055263 2580 flags.go:64] FLAG: --hostname-override="ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055266 2580 flags.go:64] FLAG: --housekeeping-interval="10s" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055269 2580 flags.go:64] FLAG: --http-check-frequency="20s" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055272 2580 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055276 2580 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055279 2580 flags.go:64] FLAG: --image-gc-high-threshold="85" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055283 2580 flags.go:64] FLAG: --image-gc-low-threshold="80" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055286 2580 flags.go:64] FLAG: --image-service-endpoint="" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055288 2580 flags.go:64] FLAG: --kernel-memcg-notification="false" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055291 2580 flags.go:64] FLAG: --kube-api-burst="100" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055294 2580 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055297 2580 flags.go:64] FLAG: --kube-api-qps="50" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055301 2580 flags.go:64] FLAG: --kube-reserved="" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055304 2580 flags.go:64] FLAG: --kube-reserved-cgroup="" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055307 2580 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055310 2580 flags.go:64] FLAG: --kubelet-cgroups="" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055312 2580 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055315 2580 flags.go:64] FLAG: --lock-file="" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055318 2580 flags.go:64] FLAG: --log-cadvisor-usage="false" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055321 2580 flags.go:64] FLAG: --log-flush-frequency="5s" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055324 2580 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jul 01 16:57:57.130607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055329 2580 flags.go:64] FLAG: --log-json-split-stream="false" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055332 2580 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055336 2580 flags.go:64] FLAG: --log-text-split-stream="false" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055339 2580 flags.go:64] FLAG: --logging-format="text" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055342 2580 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055345 2580 flags.go:64] FLAG: --make-iptables-util-chains="true" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055348 2580 flags.go:64] FLAG: --manifest-url="" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055350 2580 flags.go:64] FLAG: --manifest-url-header="" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055355 2580 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055358 2580 flags.go:64] FLAG: --max-open-files="1000000" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055362 2580 flags.go:64] FLAG: --max-pods="110" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055365 2580 flags.go:64] FLAG: --maximum-dead-containers="-1" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055368 2580 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055371 2580 flags.go:64] FLAG: --memory-manager-policy="None" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055374 2580 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055377 2580 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055379 2580 flags.go:64] FLAG: --node-ip="0.0.0.0" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055384 2580 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055391 2580 flags.go:64] FLAG: --node-status-max-images="50" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055394 2580 flags.go:64] FLAG: --node-status-update-frequency="10s" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055396 2580 flags.go:64] FLAG: --oom-score-adj="-999" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055399 2580 flags.go:64] FLAG: --pod-cidr="" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055403 2580 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:8a2632cd31d78133be7b5fb6999469b2acf9b98d077c7f6ae5d0d94083d76b04" Jul 01 16:57:57.135262 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055408 2580 flags.go:64] FLAG: --pod-manifest-path="" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055422 2580 flags.go:64] FLAG: --pod-max-pids="-1" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055426 2580 flags.go:64] FLAG: --pods-per-core="0" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055430 2580 flags.go:64] FLAG: --port="10250" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055435 2580 flags.go:64] FLAG: --protect-kernel-defaults="false" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055439 2580 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-024902f48d26853aa" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055444 2580 flags.go:64] FLAG: --qos-reserved="" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055455 2580 flags.go:64] FLAG: --read-only-port="10255" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055471 2580 flags.go:64] FLAG: --register-node="true" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055475 2580 flags.go:64] FLAG: --register-with-taints="" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055480 2580 flags.go:64] FLAG: --registry-burst="10" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055483 2580 flags.go:64] FLAG: --registry-qps="5" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055489 2580 flags.go:64] FLAG: --reserved-cpus="" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055492 2580 flags.go:64] FLAG: --reserved-memory="" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055495 2580 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055499 2580 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055502 2580 flags.go:64] FLAG: --rotate-certificates="false" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055505 2580 flags.go:64] FLAG: --rotate-server-certificates="false" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055508 2580 flags.go:64] FLAG: --runonce="false" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055511 2580 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055514 2580 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055517 2580 flags.go:64] FLAG: --seccomp-default="false" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055520 2580 flags.go:64] FLAG: --serialize-image-pulls="true" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055522 2580 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055526 2580 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055529 2580 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jul 01 16:57:57.136565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055536 2580 flags.go:64] FLAG: --storage-driver-password="root" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055539 2580 flags.go:64] FLAG: --storage-driver-secure="false" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055542 2580 flags.go:64] FLAG: --storage-driver-table="stats" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055544 2580 flags.go:64] FLAG: --storage-driver-user="root" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055547 2580 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055551 2580 flags.go:64] FLAG: --sync-frequency="1m0s" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055554 2580 flags.go:64] FLAG: --system-cgroups="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055557 2580 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055563 2580 flags.go:64] FLAG: --system-reserved-cgroup="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055566 2580 flags.go:64] FLAG: --tls-cert-file="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055569 2580 flags.go:64] FLAG: --tls-cipher-suites="[]" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055573 2580 flags.go:64] FLAG: --tls-min-version="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055576 2580 flags.go:64] FLAG: --tls-private-key-file="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055579 2580 flags.go:64] FLAG: --topology-manager-policy="none" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055582 2580 flags.go:64] FLAG: --topology-manager-policy-options="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055585 2580 flags.go:64] FLAG: --topology-manager-scope="container" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055588 2580 flags.go:64] FLAG: --v="2" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055592 2580 flags.go:64] FLAG: --version="false" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055602 2580 flags.go:64] FLAG: --vmodule="" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055606 2580 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.055609 2580 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055732 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055736 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055739 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.144169 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055742 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055745 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055748 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055751 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055754 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055757 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055760 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055762 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055766 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055769 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055771 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055774 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055777 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055779 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055782 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055784 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055787 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055790 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055793 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055795 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.145354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055798 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055814 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055817 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055820 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055823 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055825 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055831 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055834 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055838 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055840 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055843 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055846 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055848 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055851 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055854 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055856 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055859 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055863 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055867 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.146208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055871 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055876 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055879 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055881 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055884 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055887 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055890 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055893 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055896 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055899 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055902 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055905 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055908 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055910 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055913 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055915 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055918 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055920 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055923 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055927 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.147065 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055929 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055932 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055934 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055937 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055940 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055942 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055945 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055948 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055950 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055953 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055955 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055958 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055960 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055964 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055966 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055969 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055971 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055974 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055976 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.147983 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055979 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055982 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055984 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055987 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055989 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055992 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055995 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.055997 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056000 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056002 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056005 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056008 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056013 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056016 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056019 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056022 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056024 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.148789 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.056027 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.056614 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.064454 2580 server.go:529] "Kubelet version" kubeletVersion="v1.34.8" Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.064471 2580 server.go:531] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.064489 2580 watchdog_linux.go:95] "Systemd watchdog is not enabled" Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.064495 2580 watchdog_linux.go:137] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064554 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064559 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064563 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064566 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064570 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064574 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064578 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064581 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.149656 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064583 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064586 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064589 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064593 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064596 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064599 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064602 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064605 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064608 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064611 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064614 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064617 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064619 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064622 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064625 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064627 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064630 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064633 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064636 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064640 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.150494 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064642 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064645 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064647 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064650 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064653 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064655 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064658 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064660 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064663 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064666 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064668 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064670 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064673 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064676 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064678 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064681 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064683 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064686 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064695 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064698 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.151357 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064700 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064704 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064708 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064711 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064714 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064717 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064720 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064722 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064725 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064728 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064731 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064734 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064736 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064739 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064742 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064744 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064747 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064750 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064752 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.152239 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064755 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064758 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064761 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064764 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064766 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064769 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064771 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064774 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064777 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064779 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064782 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064784 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064792 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064795 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064798 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064820 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064823 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064826 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064830 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.153123 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064833 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064836 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064839 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064842 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064845 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064848 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064851 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064854 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064856 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064859 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064862 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064865 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064867 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.064873 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064989 2580 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jul 01 16:57:57.154002 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064994 2580 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.064997 2580 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065000 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065003 2580 feature_gate.go:328] unrecognized feature gate: ProvisioningRequestAvailable Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065006 2580 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065008 2580 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065011 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065014 2580 feature_gate.go:328] unrecognized feature gate: AzureDualStackInstall Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065016 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065020 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065022 2580 feature_gate.go:328] unrecognized feature gate: SignatureStores Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065034 2580 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065036 2580 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065039 2580 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065042 2580 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065044 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065047 2580 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065049 2580 feature_gate.go:328] unrecognized feature gate: KMSEncryption Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065052 2580 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065055 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jul 01 16:57:57.296639 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065057 2580 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065062 2580 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065065 2580 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065068 2580 feature_gate.go:328] unrecognized feature gate: DualReplica Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065070 2580 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065075 2580 feature_gate.go:328] unrecognized feature gate: Example2 Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065077 2580 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065079 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagement Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065082 2580 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterInstall Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065087 2580 feature_gate.go:351] Setting GA feature gate VolumeAttributesClass=true. It will be removed in a future release. Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065090 2580 feature_gate.go:328] unrecognized feature gate: CRDCompatibilityRequirementOperator Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065093 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIMachineManagementVSphere Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065096 2580 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065099 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065102 2580 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065104 2580 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065107 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065109 2580 feature_gate.go:328] unrecognized feature gate: NewOLM Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065112 2580 feature_gate.go:328] unrecognized feature gate: OVNObservability Jul 01 16:57:57.312596 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065114 2580 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065117 2580 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065119 2580 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065122 2580 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065124 2580 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065128 2580 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065131 2580 feature_gate.go:328] unrecognized feature gate: EventTTL Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065134 2580 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065136 2580 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065139 2580 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065142 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesCPMS Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065144 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065146 2580 feature_gate.go:328] unrecognized feature gate: PinnedImages Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065149 2580 feature_gate.go:328] unrecognized feature gate: AWSDualStackInstall Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065153 2580 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065156 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065158 2580 feature_gate.go:328] unrecognized feature gate: OSStreams Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065161 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065165 2580 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jul 01 16:57:57.313463 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065169 2580 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065172 2580 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065175 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIWithoutOLM Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065178 2580 feature_gate.go:328] unrecognized feature gate: NewOLMBoxCutterRuntime Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065180 2580 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065183 2580 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065185 2580 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065188 2580 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065191 2580 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065193 2580 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065196 2580 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065198 2580 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065201 2580 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065203 2580 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065206 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065208 2580 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065211 2580 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065213 2580 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065216 2580 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065219 2580 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jul 01 16:57:57.314257 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065222 2580 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065225 2580 feature_gate.go:328] unrecognized feature gate: HyperShiftOnlyDynamicResourceAllocation Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065227 2580 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065230 2580 feature_gate.go:328] unrecognized feature gate: GCPDualStackInstall Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065232 2580 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065234 2580 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065237 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065241 2580 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065243 2580 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065246 2580 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065249 2580 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065252 2580 feature_gate.go:328] unrecognized feature gate: Example Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065255 2580 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065258 2580 feature_gate.go:328] unrecognized feature gate: OnPremDNSRecords Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065261 2580 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065263 2580 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065266 2580 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065269 2580 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065271 2580 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jul 01 16:57:57.315080 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:57.065275 2580 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.065280 2580 feature_gate.go:384] feature gates: {map[CBORServingAndStorage:false ClientsAllowCBOR:false ClientsPreferCBOR:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutableCSINodeAllocatableCount:false MutatingAdmissionPolicy:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:true]} Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.065414 2580 server.go:962] "Client rotation is on, will bootstrap in background" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.067362 2580 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.068695 2580 server.go:1019] "Starting client certificate rotation" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.068796 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.068846 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.070972 2580 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.074487 2580 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.086460 2580 log.go:25] "Validated CRI v1 runtime API" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.090037 2580 log.go:25] "Validated CRI v1 image API" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.090885 2580 server.go:1429] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jul 01 16:57:57.315687 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.093340 2580 fs.go:135] Filesystem UUIDs: map[7B77-95E7:/dev/nvme0n1p2 9900fd8e-3db3-4848-8598-b241846a41fc:/dev/nvme0n1p3 fdb8e8b1-ae2b-455f-8104-56eb0a9620a4:/dev/nvme0n1p4] Jul 01 16:57:57.316097 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.093357 2580 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.099274 2580 manager.go:216] Machine: {Timestamp:2026-07-01 16:57:57.097352654 +0000 UTC m=+0.336909321 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3077828 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2730482537ee89c83aad8c04543f82 SystemUUID:ec273048-2537-ee89-c83a-ad8c04543f82 BootID:08d0db4f-f7ed-43c6-b12f-e31824648ded Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6107136 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:fb:4f:b4:c9:07 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:fb:4f:b4:c9:07 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:72:a4:6b:98:38:c9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.099704 2580 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.099911 2580 manager.go:232] Version: {KernelVersion:5.14.0-570.124.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260623-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.102781 2580 container_manager_linux.go:273] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.102826 2580 container_manager_linux.go:278] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-138-241.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.103193 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:57:57.316323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.103497 2580 topology_manager.go:138] "Creating topology manager with none policy" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.103510 2580 container_manager_linux.go:309] "Creating device plugin manager" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.103518 2580 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.104246 2580 server.go:74] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.105044 2580 container_manager_linux.go:318] "Creating Dynamic Resource Allocation (DRA) manager" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108660 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108822 2580 server.go:1240] "Using root directory" path="/var/lib/kubelet" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108858 2580 kubelet.go:486] "Attempting to sync node with API server" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108871 2580 kubelet.go:387] "Adding static pod path" path="/etc/kubernetes/manifests" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108887 2580 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108899 2580 kubelet.go:398] "Adding apiserver pod source" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.108912 2580 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.109329 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.109348 2580 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.111601 2580 kuberuntime_manager.go:301] "Container runtime initialized" containerRuntime="cri-o" version="1.34.8-3.rhaos4.21.git91b4b0b.el9" apiVersion="v1" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.113436 2580 kubelet.go:960] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.113472 2580 kubelet.go:984] "Not starting PodCertificateRequest manager because we are in static kubelet mode or the PodCertificateProjection feature gate is disabled" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114110 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114127 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114135 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114143 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114153 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114162 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114235 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114243 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114252 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114260 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114274 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.114286 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.115247 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.115260 2580 plugins.go:610] "Loaded volume plugin" pluginName="kubernetes.io/image" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.118732 2580 server.go:1268] "Started kubelet" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.119107 2580 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.120987 2580 server.go:310] "Adding debug handlers to kubelet server" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.121752 2580 ratelimit.go:56] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.121813 2580 server_v1.go:49] "podresources" method="list" useActivePods=true Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.128561 2580 server.go:249] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.132198 2580 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.132502 2580 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.133759 2580 csi_plugin.go:990] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-138-241.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.133822 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134332 2580 volume_manager.go:311] "The desired_state_of_world populator starts" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134345 2580 volume_manager.go:313] "Starting Kubelet Volume Manager" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.134536 2580 kubelet.go:1638] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jul 01 16:57:57.317482 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134676 2580 reconstruct.go:97] "Volume reconstruction finished" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134689 2580 reconciler.go:29] "Reconciler: start to sync state" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134719 2580 desired_state_of_world_populator.go:146] "Desired state populator starts to run" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134770 2580 factory.go:55] Registering systemd factory Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.134785 2580 factory.go:223] Registration of the systemd container factory successfully Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135025 2580 factory.go:153] Registering CRI-O factory Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135040 2580 factory.go:223] Registration of the crio container factory successfully Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135104 2580 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135131 2580 factory.go:103] Registering Raw factory Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135150 2580 manager.go:1196] Started watching for new ooms in manager Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.135546 2580 manager.go:349] Starting recovery of all containers Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.141421 2580 manager.go:354] Recovery completed Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.160771 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.162032 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.163227 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9xpl6" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165223 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165252 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165262 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165835 2580 cpu_manager.go:222] "Starting CPU manager" policy="none" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165847 2580 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165867 2580 state_mem.go:36] "Initialized new in-memory state store" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.165909 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.166041 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.166291 2580 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-138-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.166393 2580 reflector.go:205] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-138-241.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.166732 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.166767 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.166787 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.166021 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18be38e94acbf41b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-07-01 16:57:57.118710811 +0000 UTC m=+0.358267479,LastTimestamp:2026-07-01 16:57:57.118710811 +0000 UTC m=+0.358267479,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jul 01 16:57:57.318822 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.167384 2580 policy_none.go:49] "None policy: Start" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.167399 2580 memory_manager.go:187] "Starting memorymanager" policy="None" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.167417 2580 state_mem.go:36] "Initializing new in-memory state store" logger="Memory Manager state checkpoint" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.168560 2580 policy_none.go:47] "Start" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.169296 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18be38e94d91f5bf default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-138-241.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-07-01 16:57:57.165241791 +0000 UTC m=+0.404798459,LastTimestamp:2026-07-01 16:57:57.165241791 +0000 UTC m=+0.404798459,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.173915 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-9xpl6" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.174115 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18be38e94d922f63 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasNoDiskPressure,Message:Node ip-10-0-138-241.ec2.internal status is now: NodeHasNoDiskPressure,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-07-01 16:57:57.165256547 +0000 UTC m=+0.404813215,LastTimestamp:2026-07-01 16:57:57.165256547 +0000 UTC m=+0.404813215,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.177289 2580 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-138-241.ec2.internal.18be38e94d92525b default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-138-241.ec2.internal,UID:ip-10-0-138-241.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientPID,Message:Node ip-10-0-138-241.ec2.internal status is now: NodeHasSufficientPID,Source:EventSource{Component:kubelet,Host:ip-10-0-138-241.ec2.internal,},FirstTimestamp:2026-07-01 16:57:57.165265499 +0000 UTC m=+0.404822167,LastTimestamp:2026-07-01 16:57:57.165265499 +0000 UTC m=+0.404822167,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-138-241.ec2.internal,}" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.213982 2580 manager.go:337] "Starting Device Plugin manager" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.214024 2580 manager.go:513] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214035 2580 server.go:87] "Starting device plugin registration server" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214343 2580 eviction_manager.go:189] "Eviction manager: starting control loop" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214357 2580 container_log_manager.go:146] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214422 2580 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214509 2580 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.214519 2580 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.215150 2580 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.215220 2580 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.222114 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv4" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.223225 2580 kubelet_network_linux.go:54] "Initialized iptables rules." protocol="IPv6" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.223267 2580 status_manager.go:244] "Starting to sync pod status with apiserver" Jul 01 16:57:57.319733 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.223286 2580 kubelet.go:2448] "Starting kubelet main sync loop" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.223317 2580 kubelet.go:2472] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.225692 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.316074 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.317254 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.317299 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.317316 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.320360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.317346 2580 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.323369 2580 kubelet.go:2535] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal"] Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.323453 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.324920 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.324949 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.324959 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.326462 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.326566 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.326592 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327184 2580 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327190 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.327206 2580 kubelet_node_status.go:536] "Error updating node status, will retry" err="error getting node \"ip-10-0-138-241.ec2.internal\": node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327213 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327223 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327187 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327277 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.327287 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.328319 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.328340 2580 kubelet_node_status.go:400] "Setting node annotation to enable volume controller attach/detach" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.328953 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientMemory" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.328983 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasNoDiskPressure" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.328999 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeHasSufficientPID" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.335939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.335961 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.335979 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/400088389ad348a7c1f16971b84768a2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"400088389ad348a7c1f16971b84768a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.347264 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.364675 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-241.ec2.internal\" not found" node="ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.470152 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.368694 2580 kubelet.go:3244] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-138-241.ec2.internal\" not found" node="ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436838 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436881 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436898 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/400088389ad348a7c1f16971b84768a2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"400088389ad348a7c1f16971b84768a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436952 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436954 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/305b7cb6230afa2bdd2454d2e859f11c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal\" (UID: \"305b7cb6230afa2bdd2454d2e859f11c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.436953 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/400088389ad348a7c1f16971b84768a2-config\") pod \"kube-apiserver-proxy-ip-10-0-138-241.ec2.internal\" (UID: \"400088389ad348a7c1f16971b84768a2\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.471184 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.448184 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.675423 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.548671 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.675423 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.649026 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:57.675423 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.668625 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.675423 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:57.671982 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:57.821493 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.750112 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:58.016768 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.851018 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:58.016768 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:57.951485 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:58.052531 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.068881 2580 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.069081 2580 reflector.go:571] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:58.152718 2580 kubelet_node_status.go:454] "Error getting the current node from lister" err="node \"ip-10-0-138-241.ec2.internal\" not found" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.166766 2580 reflector.go:439] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.177538 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-30 16:52:57 +0000 UTC" deadline="2028-03-04 19:03:52.421150991 +0000 UTC" Jul 01 16:57:58.196060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.177587 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14690h5m54.243567589s" Jul 01 16:57:58.375916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.232520 2580 reflector.go:439] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:57:58.375916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.234718 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" Jul 01 16:57:58.375916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.271148 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:57:58.375916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.273010 2580 kubelet.go:3248] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" Jul 01 16:57:58.375916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.287213 2580 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jul 01 16:57:58.571456 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:58.464279 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod400088389ad348a7c1f16971b84768a2.slice/crio-b3f4b46b56401e17e92c3c11568f6bc16f8e83ca527bc605e78ba04be9913d15 WatchSource:0}: Error finding container b3f4b46b56401e17e92c3c11568f6bc16f8e83ca527bc605e78ba04be9913d15: Status 404 returned error can't find the container with id b3f4b46b56401e17e92c3c11568f6bc16f8e83ca527bc605e78ba04be9913d15 Jul 01 16:57:58.571456 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.468823 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 16:57:58.742731 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:58.663425 2580 reflector.go:439] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:57:59.071376 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.023587 2580 reflector.go:439] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jul 01 16:57:59.071376 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.031677 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod305b7cb6230afa2bdd2454d2e859f11c.slice/crio-197a2699e9acd77ff5d80e69cc12e5ddb5a0662904b2c39a03ac313ca0e5d51f WatchSource:0}: Error finding container 197a2699e9acd77ff5d80e69cc12e5ddb5a0662904b2c39a03ac313ca0e5d51f: Status 404 returned error can't find the container with id 197a2699e9acd77ff5d80e69cc12e5ddb5a0662904b2c39a03ac313ca0e5d51f Jul 01 16:57:59.111490 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.111457 2580 apiserver.go:52] "Watching apiserver" Jul 01 16:57:59.121305 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.121272 2580 reflector.go:439] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jul 01 16:57:59.121696 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.121674 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-d2bhk","openshift-network-diagnostics/network-check-target-vncqb","kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal","openshift-image-registry/node-ca-922gs","openshift-multus/multus-additional-cni-plugins-k8sxt","openshift-multus/network-metrics-daemon-tz2kn","openshift-network-operator/iptables-alerter-ngvm6","openshift-ovn-kubernetes/ovnkube-node-g8mk4","kube-system/global-pull-secret-syncer-7tknz","kube-system/konnectivity-agent-4pjmt","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4","openshift-cluster-node-tuning-operator/tuned-r69ht","openshift-dns/node-resolver-hgs2r","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal"] Jul 01 16:57:59.124690 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.124669 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.126934 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.126909 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:57:59.127062 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.127010 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:57:59.127648 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.127629 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jul 01 16:57:59.127986 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.127967 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.128073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.127973 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jul 01 16:57:59.128073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.128001 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rqcbm\"" Jul 01 16:57:59.128073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.128055 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.129312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.129296 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.131785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.131754 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.131910 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.131818 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jul 01 16:57:59.131910 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.131839 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.131910 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.131843 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.132101 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.132084 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-p8twl\"" Jul 01 16:57:59.134090 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.134073 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.134189 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.134164 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:57:59.134267 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.134254 2580 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jul 01 16:57:59.136436 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.136420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.138765 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.138748 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-ttdvv\"" Jul 01 16:57:59.139052 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.139034 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jul 01 16:57:59.139052 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.139049 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.139466 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.139449 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jul 01 16:57:59.139548 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.139485 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.140520 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.140491 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.140616 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.140524 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jul 01 16:57:59.140950 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.140929 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wz8r8\"" Jul 01 16:57:59.142931 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.142458 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jul 01 16:57:59.143102 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.143084 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.143209 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.143168 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:57:59.143335 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.143314 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.144461 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.144373 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jul 01 16:57:59.144691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.144670 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jul 01 16:57:59.144786 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.144699 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jul 01 16:57:59.144877 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.144866 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-x2l9q\"" Jul 01 16:57:59.144967 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.144949 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.146792 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.146771 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.149284 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.149234 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jul 01 16:57:59.149383 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.149337 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xssw6\"" Jul 01 16:57:59.149383 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.149363 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jul 01 16:57:59.149649 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.149635 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.152168 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.152149 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.152556 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.152533 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jul 01 16:57:59.152625 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.152555 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.152904 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.152875 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-x6s5b\"" Jul 01 16:57:59.152904 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.152875 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.154318 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.154300 2580 reflector.go:439] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jul 01 16:57:59.154682 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.154652 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.155401 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.155374 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-68dxp\"" Jul 01 16:57:59.155401 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.155382 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.155401 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.155392 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.157188 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157171 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jul 01 16:57:59.157351 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157336 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-vpd46\"" Jul 01 16:57:59.157466 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157450 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.157524 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157477 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-netns\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157524 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157496 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-system-cni-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.157603 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157534 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.157603 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157552 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-kubelet\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157603 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157568 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-cni-binary-copy\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157603 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157597 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hvb9j\" (UniqueName: \"kubernetes.io/projected/40c69be1-a415-4029-b534-1ec9ea957fef-kube-api-access-hvb9j\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157744 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:57:59.157744 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-multus\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157744 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157695 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-cnibin\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.157744 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157725 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.157908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157754 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.157908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9e41189-1f28-451e-ad86-f1e7c420c40c-host-slash\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.157908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157803 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-multus-certs\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157849 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-k8s-cni-cncf-io\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.157908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157872 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-system-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157908 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6sqc\" (UniqueName: \"kubernetes.io/projected/29b0fcf2-fa42-408f-9310-97306bbfb0de-kube-api-access-f6sqc\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157960 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.157978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9c7l6\" (UniqueName: \"kubernetes.io/projected/b9e41189-1f28-451e-ad86-f1e7c420c40c-kube-api-access-9c7l6\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158009 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-socket-dir-parent\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158038 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-bin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158064 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-hostroot\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158086 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-etc-kubernetes\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158106 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-host\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158123 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fjjjm\" (UniqueName: \"kubernetes.io/projected/c4909179-8cdd-41ef-b211-0915ff19bc87-kube-api-access-fjjjm\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-conf-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158197 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-multus-daemon-config\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158223 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-serviceca\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158242 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-99hgv\" (UniqueName: \"kubernetes.io/projected/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-kube-api-access-99hgv\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158268 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-os-release\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158293 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9e41189-1f28-451e-ad86-f1e7c420c40c-iptables-alerter-script\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158320 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158346 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-cnibin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.158407 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.158379 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-os-release\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.180108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.180083 2580 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-5g4bh" Jul 01 16:57:59.190484 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.190460 2580 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-5g4bh" Jul 01 16:57:59.229497 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.229445 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"305b7cb6230afa2bdd2454d2e859f11c","Type":"ContainerStarted","Data":"197a2699e9acd77ff5d80e69cc12e5ddb5a0662904b2c39a03ac313ca0e5d51f"} Jul 01 16:57:59.235787 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.235754 2580 desired_state_of_world_populator.go:154] "Finished populating initial desired state of world" Jul 01 16:57:59.236121 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.236083 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" event={"ID":"400088389ad348a7c1f16971b84768a2","Type":"ContainerStarted","Data":"b3f4b46b56401e17e92c3c11568f6bc16f8e83ca527bc605e78ba04be9913d15"} Jul 01 16:57:59.259029 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.258978 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-lib-modules\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.259029 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259032 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-var-lib-kubelet\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.259329 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259067 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-socket-dir-parent\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259329 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-bin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259560 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259498 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-hostroot\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259560 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259547 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.259709 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259599 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-bin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259709 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259600 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-hostroot\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259709 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259652 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-env-overrides\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.259709 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259682 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-multus-daemon-config\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.259709 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259706 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-99hgv\" (UniqueName: \"kubernetes.io/projected/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-kube-api-access-99hgv\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.260026 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259729 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9e41189-1f28-451e-ad86-f1e7c420c40c-iptables-alerter-script\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.260026 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-etc-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.260026 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259772 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-socket-dir-parent\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.260026 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.259788 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-netd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.261862 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.261582 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b9e41189-1f28-451e-ad86-f1e7c420c40c-iptables-alerter-script\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.262092 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262065 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-multus-daemon-config\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.262405 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262386 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-kubelet-config\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.262505 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-socket-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.262505 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.262505 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262497 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-cnibin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.262676 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262536 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.262676 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262571 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-systemd-units\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.262676 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262599 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-systemd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.262676 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262633 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.262676 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262671 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xpp4\" (UniqueName: \"kubernetes.io/projected/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-kube-api-access-7xpp4\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262714 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-kubernetes\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262748 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-netns\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262778 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262816 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262851 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-run\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262897 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-sys\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262935 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-kubelet\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.262990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.262971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-ovn\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263000 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovn-node-metrics-cert\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263073 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-conf\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263111 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glthb\" (UniqueName: \"kubernetes.io/projected/125c559d-74f0-44eb-bffd-7d568d0c5b59-kube-api-access-glthb\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263158 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-cnibin\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263195 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263232 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9e41189-1f28-451e-ad86-f1e7c420c40c-host-slash\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263301 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-modprobe-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263332 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p92ms\" (UniqueName: \"kubernetes.io/projected/f3e0f1c8-be07-4739-844d-7d02b51b42cc-kube-api-access-p92ms\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.263378 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-multus-certs\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263400 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-k8s-cni-cncf-io\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263433 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-slash\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263518 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-dbus\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-registration-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263670 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9dhht\" (UniqueName: \"kubernetes.io/projected/7fcfbe51-d769-4162-89c5-95531c0af39e-kube-api-access-9dhht\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.263728 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263754 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263759 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-kubelet\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.263816 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-cnibin\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.263920 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.263862 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:57:59.7637964 +0000 UTC m=+3.003353057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:57:59.264447 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264222 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-multus-certs\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.264447 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264375 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-tuning-conf-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.264447 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264435 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b9e41189-1f28-451e-ad86-f1e7c420c40c-host-slash\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.264610 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264466 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-cnibin\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.264788 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264750 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.264788 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264774 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-k8s-cni-cncf-io\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264785 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-binary-copy\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264792 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-run-netns\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264756 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-tmp\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264836 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-kubelet\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264845 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/c4909179-8cdd-41ef-b211-0915ff19bc87-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-etc-kubernetes\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.264974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.264956 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-host\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265005 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-fjjjm\" (UniqueName: \"kubernetes.io/projected/c4909179-8cdd-41ef-b211-0915ff19bc87-kube-api-access-fjjjm\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-etc-kubernetes\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265084 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-systemd\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265094 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-host\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265119 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-host\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265160 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-tuned\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265199 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e0f1c8-be07-4739-844d-7d02b51b42cc-tmp-dir\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265233 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-script-lib\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.265306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265280 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-conf-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265312 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-serviceca\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265343 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-os-release\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265388 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-netns\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265428 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-bin\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265477 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-multus-conf-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265618 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-os-release\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265666 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7b7e371-fbff-4505-897b-610cc43b146f-konnectivity-ca\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.265769 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265700 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-etc-selinux\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265794 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-os-release\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265847 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-var-lib-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265883 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7b7e371-fbff-4505-897b-610cc43b146f-agent-certs\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265898 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-serviceca\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-sys-fs\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.265971 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f3e0f1c8-be07-4739-844d-7d02b51b42cc-hosts-file\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266051 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-os-release\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266062 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-system-cni-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.266160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-cni-binary-copy\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266178 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c4909179-8cdd-41ef-b211-0915ff19bc87-system-cni-dir\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.266609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266224 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-hvb9j\" (UniqueName: \"kubernetes.io/projected/40c69be1-a415-4029-b534-1ec9ea957fef-kube-api-access-hvb9j\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266277 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:57:59.266609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266391 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-log-socket\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.266609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266507 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-multus\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266845 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266783 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-host-var-lib-cni-multus\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.266959 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.266910 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/40c69be1-a415-4029-b534-1ec9ea957fef-cni-binary-copy\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.267251 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267229 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-node-log\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.267689 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267658 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.267784 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267702 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-system-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.267784 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267733 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-f6sqc\" (UniqueName: \"kubernetes.io/projected/29b0fcf2-fa42-408f-9310-97306bbfb0de-kube-api-access-f6sqc\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.267784 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267760 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-device-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.267956 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267786 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9c7l6\" (UniqueName: \"kubernetes.io/projected/b9e41189-1f28-451e-ad86-f1e7c420c40c-kube-api-access-9c7l6\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.267956 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267813 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-config\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.267956 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267838 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysconfig\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.268106 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.267986 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/40c69be1-a415-4029-b534-1ec9ea957fef-system-cni-dir\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.271058 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.270863 2580 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jul 01 16:57:59.275646 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.275596 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-99hgv\" (UniqueName: \"kubernetes.io/projected/13204f90-24a2-4ee1-83c1-e6a31a5a76b8-kube-api-access-99hgv\") pod \"node-ca-922gs\" (UID: \"13204f90-24a2-4ee1-83c1-e6a31a5a76b8\") " pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.278172 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.278139 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fjjjm\" (UniqueName: \"kubernetes.io/projected/c4909179-8cdd-41ef-b211-0915ff19bc87-kube-api-access-fjjjm\") pod \"multus-additional-cni-plugins-k8sxt\" (UID: \"c4909179-8cdd-41ef-b211-0915ff19bc87\") " pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.279545 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.279520 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hvb9j\" (UniqueName: \"kubernetes.io/projected/40c69be1-a415-4029-b534-1ec9ea957fef-kube-api-access-hvb9j\") pod \"multus-d2bhk\" (UID: \"40c69be1-a415-4029-b534-1ec9ea957fef\") " pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.279665 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.279572 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6sqc\" (UniqueName: \"kubernetes.io/projected/29b0fcf2-fa42-408f-9310-97306bbfb0de-kube-api-access-f6sqc\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.279918 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.279843 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:57:59.279918 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.279871 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:57:59.280082 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.279940 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:57:59.280082 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.280028 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:57:59.780009652 +0000 UTC m=+3.019566327 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:57:59.282556 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.282535 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9c7l6\" (UniqueName: \"kubernetes.io/projected/b9e41189-1f28-451e-ad86-f1e7c420c40c-kube-api-access-9c7l6\") pod \"iptables-alerter-ngvm6\" (UID: \"b9e41189-1f28-451e-ad86-f1e7c420c40c\") " pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.368661 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368624 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-kubelet\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368673 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-tmp\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368704 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368731 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-systemd\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368754 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-host\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-tuned\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e0f1c8-be07-4739-844d-7d02b51b42cc-tmp-dir\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.368875 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-script-lib\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368887 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-netns\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368917 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-bin\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7b7e371-fbff-4505-897b-610cc43b146f-konnectivity-ca\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368972 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-systemd\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.368978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-etc-selinux\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-var-lib-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369076 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7b7e371-fbff-4505-897b-610cc43b146f-agent-certs\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369101 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-sys-fs\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369124 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f3e0f1c8-be07-4739-844d-7d02b51b42cc-hosts-file\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-etc-selinux\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369167 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-log-socket\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369188 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-kubelet\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369213 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-log-socket\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369216 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-node-log\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.369312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369261 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-var-lib-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-device-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369338 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-config\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369363 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysconfig\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369388 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-lib-modules\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-var-lib-kubelet\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369475 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-env-overrides\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369504 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-etc-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369530 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-netd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-device-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369575 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-kubelet-config\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-socket-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.370074 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369607 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-netns\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369651 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-systemd-units\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369682 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-run-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369678 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-systemd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369724 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xpp4\" (UniqueName: \"kubernetes.io/projected/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-kube-api-access-7xpp4\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-kubernetes\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369792 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/b7b7e371-fbff-4505-897b-610cc43b146f-konnectivity-ca\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369805 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369792 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-host\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369836 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-run\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369835 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-script-lib\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369849 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-bin\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369883 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-sys-fs\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369888 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-sys\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.370825 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.369931 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-sys\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369962 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-ovn\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.369982 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:57:59.869967971 +0000 UTC m=+3.109524631 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369996 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-systemd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370004 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-cni-netd\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370068 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-etc-openvswitch\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370117 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-systemd-units\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370124 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-socket-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369652 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-node-log\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370139 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-env-overrides\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/f3e0f1c8-be07-4739-844d-7d02b51b42cc-tmp-dir\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370197 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-kubelet-config\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.371691 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.369930 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-run-ovn\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370226 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovn-node-metrics-cert\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370245 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370253 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-var-lib-kubelet\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370265 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-lib-modules\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-kubernetes\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370299 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-run\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370307 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370348 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370378 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-conf\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370407 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-glthb\" (UniqueName: \"kubernetes.io/projected/125c559d-74f0-44eb-bffd-7d568d0c5b59-kube-api-access-glthb\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370428 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370438 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-modprobe-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370464 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-p92ms\" (UniqueName: \"kubernetes.io/projected/f3e0f1c8-be07-4739-844d-7d02b51b42cc-kube-api-access-p92ms\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.372419 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370474 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/f3e0f1c8-be07-4739-844d-7d02b51b42cc-hosts-file\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-slash\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370519 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-dbus\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370543 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-registration-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370565 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-9dhht\" (UniqueName: \"kubernetes.io/projected/7fcfbe51-d769-4162-89c5-95531c0af39e-kube-api-access-9dhht\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370595 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-modprobe-d\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370519 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysctl-conf\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370670 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-host-slash\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370379 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-sysconfig\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370724 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ffc63e3c-606c-44ad-abd1-d8c725769df3-dbus\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370759 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/7fcfbe51-d769-4162-89c5-95531c0af39e-registration-dir\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.370764 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovnkube-config\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.373142 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.373102 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-ovn-node-metrics-cert\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.374306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.374184 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-etc-tuned\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.374624 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.374578 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/125c559d-74f0-44eb-bffd-7d568d0c5b59-tmp\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.376998 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.376946 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/b7b7e371-fbff-4505-897b-610cc43b146f-agent-certs\") pod \"konnectivity-agent-4pjmt\" (UID: \"b7b7e371-fbff-4505-897b-610cc43b146f\") " pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.383027 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.382982 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xpp4\" (UniqueName: \"kubernetes.io/projected/9ded3f35-ecd0-4c5f-8873-7ed0743a66d9-kube-api-access-7xpp4\") pod \"ovnkube-node-g8mk4\" (UID: \"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9\") " pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.388498 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.388451 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glthb\" (UniqueName: \"kubernetes.io/projected/125c559d-74f0-44eb-bffd-7d568d0c5b59-kube-api-access-glthb\") pod \"tuned-r69ht\" (UID: \"125c559d-74f0-44eb-bffd-7d568d0c5b59\") " pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.389838 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.389802 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p92ms\" (UniqueName: \"kubernetes.io/projected/f3e0f1c8-be07-4739-844d-7d02b51b42cc-kube-api-access-p92ms\") pod \"node-resolver-hgs2r\" (UID: \"f3e0f1c8-be07-4739-844d-7d02b51b42cc\") " pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.391333 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.391302 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9dhht\" (UniqueName: \"kubernetes.io/projected/7fcfbe51-d769-4162-89c5-95531c0af39e-kube-api-access-9dhht\") pod \"aws-ebs-csi-driver-node-htpg4\" (UID: \"7fcfbe51-d769-4162-89c5-95531c0af39e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.440532 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.440502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-d2bhk" Jul 01 16:57:59.447701 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.447640 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-922gs" Jul 01 16:57:59.459848 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.459803 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" Jul 01 16:57:59.460354 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.460323 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod13204f90_24a2_4ee1_83c1_e6a31a5a76b8.slice/crio-74e6ee8cce7b446b8f10729042e243b15a9cd7829710cab1b064a9ebe9ef1f08 WatchSource:0}: Error finding container 74e6ee8cce7b446b8f10729042e243b15a9cd7829710cab1b064a9ebe9ef1f08: Status 404 returned error can't find the container with id 74e6ee8cce7b446b8f10729042e243b15a9cd7829710cab1b064a9ebe9ef1f08 Jul 01 16:57:59.466166 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.466138 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-ngvm6" Jul 01 16:57:59.472137 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.471925 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:57:59.472544 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.472509 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc4909179_8cdd_41ef_b211_0915ff19bc87.slice/crio-1f441b8a27e066833211460b83d9fab1ea782f15eaffb09b881e46fa13228b7f WatchSource:0}: Error finding container 1f441b8a27e066833211460b83d9fab1ea782f15eaffb09b881e46fa13228b7f: Status 404 returned error can't find the container with id 1f441b8a27e066833211460b83d9fab1ea782f15eaffb09b881e46fa13228b7f Jul 01 16:57:59.477999 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.477919 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:57:59.478634 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.478605 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb9e41189_1f28_451e_ad86_f1e7c420c40c.slice/crio-0ec6979d334d509cec0da1d2a1f25cca6bee370083a9e3b3f62117174a017615 WatchSource:0}: Error finding container 0ec6979d334d509cec0da1d2a1f25cca6bee370083a9e3b3f62117174a017615: Status 404 returned error can't find the container with id 0ec6979d334d509cec0da1d2a1f25cca6bee370083a9e3b3f62117174a017615 Jul 01 16:57:59.484119 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.484096 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" Jul 01 16:57:59.484920 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.484869 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9ded3f35_ecd0_4c5f_8873_7ed0743a66d9.slice/crio-2a5d4804714a307fd6f606783feccda7fd19546db7b96f296322b54595d2aae1 WatchSource:0}: Error finding container 2a5d4804714a307fd6f606783feccda7fd19546db7b96f296322b54595d2aae1: Status 404 returned error can't find the container with id 2a5d4804714a307fd6f606783feccda7fd19546db7b96f296322b54595d2aae1 Jul 01 16:57:59.489208 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.489131 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-r69ht" Jul 01 16:57:59.491208 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.491174 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7b7e371_fbff_4505_897b_610cc43b146f.slice/crio-9c3ceb7087411635a7e4cf038a5fe7b6372791e432c8a21e3c7a3aa1e4b65b2c WatchSource:0}: Error finding container 9c3ceb7087411635a7e4cf038a5fe7b6372791e432c8a21e3c7a3aa1e4b65b2c: Status 404 returned error can't find the container with id 9c3ceb7087411635a7e4cf038a5fe7b6372791e432c8a21e3c7a3aa1e4b65b2c Jul 01 16:57:59.494297 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.494092 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-hgs2r" Jul 01 16:57:59.505702 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.505457 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod125c559d_74f0_44eb_bffd_7d568d0c5b59.slice/crio-15a8de4d14b0b7b8b5f626a3f41cd8faf3e19becc108b1e9694c73dab417c54d WatchSource:0}: Error finding container 15a8de4d14b0b7b8b5f626a3f41cd8faf3e19becc108b1e9694c73dab417c54d: Status 404 returned error can't find the container with id 15a8de4d14b0b7b8b5f626a3f41cd8faf3e19becc108b1e9694c73dab417c54d Jul 01 16:57:59.512793 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:57:59.512759 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf3e0f1c8_be07_4739_844d_7d02b51b42cc.slice/crio-997cb15536e66ed3cb14c4fa9b2f369f0276cd7092ec421af8c37aa5d372e427 WatchSource:0}: Error finding container 997cb15536e66ed3cb14c4fa9b2f369f0276cd7092ec421af8c37aa5d372e427: Status 404 returned error can't find the container with id 997cb15536e66ed3cb14c4fa9b2f369f0276cd7092ec421af8c37aa5d372e427 Jul 01 16:57:59.774749 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.774643 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:57:59.774950 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.774771 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:57:59.774950 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.774858 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:58:00.774841429 +0000 UTC m=+4.014398085 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:57:59.875796 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.875740 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:57:59.875796 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:57:59.875804 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:57:59.876090 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.875986 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:57:59.876090 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.876012 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:57:59.876090 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.876025 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:57:59.876090 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.876082 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:57:59.876284 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.876096 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:00.876075745 +0000 UTC m=+4.115632425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:57:59.876284 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:57:59.876168 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:00.876148546 +0000 UTC m=+4.115705211 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:00.193579 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.193467 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:52:59 +0000 UTC" deadline="2028-03-29 07:34:38.38055377 +0000 UTC" Jul 01 16:58:00.193579 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.193530 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15278h36m38.187029001s" Jul 01 16:58:00.223583 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.223553 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:00.223826 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.223676 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:00.241058 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.241017 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hgs2r" event={"ID":"f3e0f1c8-be07-4739-844d-7d02b51b42cc","Type":"ContainerStarted","Data":"997cb15536e66ed3cb14c4fa9b2f369f0276cd7092ec421af8c37aa5d372e427"} Jul 01 16:58:00.242541 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.242504 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r69ht" event={"ID":"125c559d-74f0-44eb-bffd-7d568d0c5b59","Type":"ContainerStarted","Data":"15a8de4d14b0b7b8b5f626a3f41cd8faf3e19becc108b1e9694c73dab417c54d"} Jul 01 16:58:00.244254 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.244225 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" event={"ID":"7fcfbe51-d769-4162-89c5-95531c0af39e","Type":"ContainerStarted","Data":"b21ff9780ae50684744c6a0ef3fad4814ee8c07ad300b51f6151ef5765185673"} Jul 01 16:58:00.245776 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.245747 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ngvm6" event={"ID":"b9e41189-1f28-451e-ad86-f1e7c420c40c","Type":"ContainerStarted","Data":"0ec6979d334d509cec0da1d2a1f25cca6bee370083a9e3b3f62117174a017615"} Jul 01 16:58:00.247443 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.247375 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-922gs" event={"ID":"13204f90-24a2-4ee1-83c1-e6a31a5a76b8","Type":"ContainerStarted","Data":"74e6ee8cce7b446b8f10729042e243b15a9cd7829710cab1b064a9ebe9ef1f08"} Jul 01 16:58:00.248982 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.248910 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4pjmt" event={"ID":"b7b7e371-fbff-4505-897b-610cc43b146f","Type":"ContainerStarted","Data":"9c3ceb7087411635a7e4cf038a5fe7b6372791e432c8a21e3c7a3aa1e4b65b2c"} Jul 01 16:58:00.250386 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.250348 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"2a5d4804714a307fd6f606783feccda7fd19546db7b96f296322b54595d2aae1"} Jul 01 16:58:00.257798 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.257761 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerStarted","Data":"1f441b8a27e066833211460b83d9fab1ea782f15eaffb09b881e46fa13228b7f"} Jul 01 16:58:00.260030 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.259978 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2bhk" event={"ID":"40c69be1-a415-4029-b534-1ec9ea957fef","Type":"ContainerStarted","Data":"b8497ecf7841c9f14ebd8c10f20f7334bd200afae3b81b377c79b9af8ec344fc"} Jul 01 16:58:00.784326 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.784241 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:00.784575 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.784418 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:00.784575 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.784485 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:58:02.784464119 +0000 UTC m=+6.024020782 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:00.884752 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.884705 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:00.884982 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:00.884785 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:00.884982 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.884967 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:00.885099 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.884986 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:00.885099 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.885001 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:00.885099 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.885066 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:02.88504646 +0000 UTC m=+6.124603132 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:00.885514 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.885495 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:00.885599 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:00.885550 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:02.885534516 +0000 UTC m=+6.125091181 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:01.195029 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:01.194979 2580 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-30 16:52:59 +0000 UTC" deadline="2028-03-02 02:29:48.890973553 +0000 UTC" Jul 01 16:58:01.195029 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:01.195029 2580 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14625h31m47.695950283s" Jul 01 16:58:01.226573 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:01.226031 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:01.226573 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:01.226098 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:01.226573 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:01.226206 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:01.226573 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:01.226470 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:02.223498 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.223458 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:02.224159 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.223617 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:02.274040 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.273338 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" event={"ID":"400088389ad348a7c1f16971b84768a2","Type":"ContainerStarted","Data":"2cfc923a36c1889001c22070cde796577e75507a9bd94f4533162e60993c683a"} Jul 01 16:58:02.282591 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.282141 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"305b7cb6230afa2bdd2454d2e859f11c","Type":"ContainerStarted","Data":"76f167e844d9b2636ab37d5c9b3cc9a818cfb3752ddadc60c663671be6c552ff"} Jul 01 16:58:02.313841 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.313295 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-138-241.ec2.internal" podStartSLOduration=4.313271092 podStartE2EDuration="4.313271092s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:02.295607523 +0000 UTC m=+5.535164209" watchObservedRunningTime="2026-07-01 16:58:02.313271092 +0000 UTC m=+5.552827773" Jul 01 16:58:02.804031 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.803991 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:02.804223 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.804184 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:02.804302 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.804253 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:58:06.804232262 +0000 UTC m=+10.043788934 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:02.905379 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.905335 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:02.905570 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:02.905413 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:02.905649 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.905598 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:02.905649 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.905615 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:02.905649 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.905626 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:02.905808 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.905688 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:06.90566842 +0000 UTC m=+10.145225113 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:02.906110 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.906085 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:02.906206 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:02.906159 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:06.90613879 +0000 UTC m=+10.145695457 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:03.224000 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:03.223964 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:03.224495 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:03.224145 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:03.224587 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:03.224568 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:03.224701 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:03.224678 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:04.224076 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:04.224017 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:04.224604 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:04.224175 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:05.228601 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:05.227914 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:05.228601 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:05.228048 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:05.228601 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:05.228128 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:05.228601 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:05.228222 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:05.294762 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:05.293994 2580 generic.go:356] "Generic (PLEG): container finished" podID="305b7cb6230afa2bdd2454d2e859f11c" containerID="76f167e844d9b2636ab37d5c9b3cc9a818cfb3752ddadc60c663671be6c552ff" exitCode=0 Jul 01 16:58:05.294762 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:05.294058 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"305b7cb6230afa2bdd2454d2e859f11c","Type":"ContainerDied","Data":"76f167e844d9b2636ab37d5c9b3cc9a818cfb3752ddadc60c663671be6c552ff"} Jul 01 16:58:06.224447 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:06.224407 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:06.224643 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.224620 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:06.842196 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:06.842075 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:06.842196 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.842190 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:06.842731 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.842266 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:58:14.842239257 +0000 UTC m=+18.081795917 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:06.943122 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:06.942919 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:06.943122 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:06.942980 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:06.943122 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943101 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:06.943122 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943114 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:06.943455 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943138 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:06.943455 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943153 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:06.943455 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943180 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:14.943157577 +0000 UTC m=+18.182714233 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:06.943455 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:06.943202 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:14.943187587 +0000 UTC m=+18.182744248 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:07.226662 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:07.226302 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:07.226662 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:07.226408 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:07.226662 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:07.226452 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:07.226662 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:07.226563 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:08.223951 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:08.223500 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:08.223951 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:08.223642 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:09.223936 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:09.223906 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:09.224136 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:09.224031 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:09.224477 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:09.224418 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:09.224700 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:09.224531 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:10.224229 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:10.224195 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:10.224678 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:10.224338 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:11.223904 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:11.223854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:11.224076 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:11.223860 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:11.224076 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:11.224035 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:11.224206 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:11.224125 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:12.223724 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:12.223696 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:12.224174 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:12.223798 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:13.224524 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:13.224488 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:13.225084 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:13.224502 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:13.225084 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:13.224613 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:13.225084 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:13.224685 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:14.224481 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:14.224444 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:14.224697 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:14.224588 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:14.907810 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:14.907750 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:14.907981 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:14.907936 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:14.908036 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:14.908016 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:58:30.907994855 +0000 UTC m=+34.147551525 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:15.008766 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:15.008723 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:15.008967 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:15.008780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:15.008967 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.008905 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:15.008967 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.008934 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:15.008967 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.008957 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:15.008967 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.008970 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:15.009198 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.008983 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:31.008962898 +0000 UTC m=+34.248519565 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:15.009198 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.009017 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:31.009002008 +0000 UTC m=+34.248558678 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:15.223706 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:15.223677 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:15.223918 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:15.223680 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:15.223918 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.223822 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:15.224031 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:15.223913 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:16.224483 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:16.224451 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:16.224955 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:16.224571 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:17.225551 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:17.225256 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:17.225551 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:17.225378 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:17.225551 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:17.225434 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:17.225551 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:17.225519 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:18.223925 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:18.223884 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:18.224098 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:18.224012 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:19.224524 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.224460 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:19.224923 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.224463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:19.224923 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:19.224581 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:19.224923 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:19.224652 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:19.324474 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.324228 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" event={"ID":"305b7cb6230afa2bdd2454d2e859f11c","Type":"ContainerStarted","Data":"439876ecf8b04bb412cadd9e3f448399797e9a94963fe3aa914eed76d80128bd"} Jul 01 16:58:19.326267 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.326197 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-hgs2r" event={"ID":"f3e0f1c8-be07-4739-844d-7d02b51b42cc","Type":"ContainerStarted","Data":"1b2ead206adf59f027a914c476e0690b9f11d7af26e352fa65356d50dc23eb6e"} Jul 01 16:58:19.327798 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.327779 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" event={"ID":"7fcfbe51-d769-4162-89c5-95531c0af39e","Type":"ContainerStarted","Data":"2a215200eb2cd19d47ede2e22bd8a97e6deb79435045a97bb198ce3b605cd232"} Jul 01 16:58:19.329506 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.329485 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-922gs" event={"ID":"13204f90-24a2-4ee1-83c1-e6a31a5a76b8","Type":"ContainerStarted","Data":"5947005e71b7fec878b403886f7941a3c9f84e4e44f200605060301c1f764af2"} Jul 01 16:58:19.342425 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.342383 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-138-241.ec2.internal" podStartSLOduration=21.342369479 podStartE2EDuration="21.342369479s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:19.342102705 +0000 UTC m=+22.581659384" watchObservedRunningTime="2026-07-01 16:58:19.342369479 +0000 UTC m=+22.581926156" Jul 01 16:58:19.360668 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.360611 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/node-resolver-hgs2r" podStartSLOduration=1.8723260160000001 podStartE2EDuration="21.360594338s" totalImagesPullingTime="19.488268322s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:19.360156993 +0000 UTC m=+22.599713670" watchObservedRunningTime="2026-07-01 16:58:19.360594338 +0000 UTC m=+22.600151015" Jul 01 16:58:19.382082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:19.382040 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/node-ca-922gs" podStartSLOduration=6.402830869 podStartE2EDuration="21.382025444s" totalImagesPullingTime="14.979194575s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:19.38172083 +0000 UTC m=+22.621277508" watchObservedRunningTime="2026-07-01 16:58:19.382025444 +0000 UTC m=+22.621582120" Jul 01 16:58:20.223784 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.223748 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:20.224015 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:20.223900 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:20.334425 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.334386 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-r69ht" event={"ID":"125c559d-74f0-44eb-bffd-7d568d0c5b59","Type":"ContainerStarted","Data":"0ddfbae4c8f52a16b92c8f93adf074c1e49538a12e8b88b89640201a8720008c"} Jul 01 16:58:20.336117 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.336085 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-4pjmt" event={"ID":"b7b7e371-fbff-4505-897b-610cc43b146f","Type":"ContainerStarted","Data":"ff624be3fd814a8366021dc169ad6b9a1acd2ca55b3e20ab0757dd7fdcb5f91f"} Jul 01 16:58:20.338684 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.338650 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"df3c3a5e925ac973a7012ce0fe2842cc8f53d331d3653364a2236ffb58cac679"} Jul 01 16:58:20.338821 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.338738 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"8b07cc11d0ec1935c78c2448345b6c5a4c1270bb5078cdeb49c3e564e36a576a"} Jul 01 16:58:20.340209 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.340185 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="d00e82f0da8bc5c4831887bcec535c2b11a67be263a6a629647f730e134e69b8" exitCode=0 Jul 01 16:58:20.340290 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.340263 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"d00e82f0da8bc5c4831887bcec535c2b11a67be263a6a629647f730e134e69b8"} Jul 01 16:58:20.341645 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.341627 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-d2bhk" event={"ID":"40c69be1-a415-4029-b534-1ec9ea957fef","Type":"ContainerStarted","Data":"44151f28445b21e3c9eb6dfe3e0ebbfe525e31159f8a69fce806b870da41e9a9"} Jul 01 16:58:20.358793 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.358749 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-r69ht" podStartSLOduration=2.863748362 podStartE2EDuration="22.358735848s" totalImagesPullingTime="19.494987486s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:20.358638116 +0000 UTC m=+23.598194796" watchObservedRunningTime="2026-07-01 16:58:20.358735848 +0000 UTC m=+23.598292522" Jul 01 16:58:20.425166 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.424528 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-d2bhk" podStartSLOduration=2.624935209 podStartE2EDuration="22.424505072s" totalImagesPullingTime="19.799569863s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:20.38839325 +0000 UTC m=+23.627949928" watchObservedRunningTime="2026-07-01 16:58:20.424505072 +0000 UTC m=+23.664061753" Jul 01 16:58:20.477395 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:20.477363 2580 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jul 01 16:58:21.223774 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.223727 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:21.223774 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.223743 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:21.224161 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:21.223895 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:21.224161 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:21.224025 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:21.227443 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.227247 2580 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-07-01T16:58:20.477387748Z","UUID":"26de351a-e647-400e-a664-640217e19750","Handler":null,"Name":"","Endpoint":""} Jul 01 16:58:21.229082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.229060 2580 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jul 01 16:58:21.229236 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.229103 2580 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jul 01 16:58:21.347031 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.346984 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"dad00c49ddf4c517ed6116b65f0d99b564e3d1486c00fb21340b3a3728827d92"} Jul 01 16:58:21.347607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.347074 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"e1b3eea93345b75847dd5c0d144602d89f575aa6f43d91225e45699f4f697ee2"} Jul 01 16:58:21.347607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.347121 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"47ad6a7c21172bf56c2a115711ed5451eb17bd7754f6f3bb52d69f957cd4772c"} Jul 01 16:58:21.347607 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.347164 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"be5a7aa9246813cb1cfc8c2e50ae31430970fcf63db7671766aab01c4e7efa35"} Jul 01 16:58:21.349100 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.349077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" event={"ID":"7fcfbe51-d769-4162-89c5-95531c0af39e","Type":"ContainerStarted","Data":"41178de648c4a88c4d761addb5fc235e3170db5f564c8a4dfbc833f3e03debda"} Jul 01 16:58:21.350748 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.350718 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-ngvm6" event={"ID":"b9e41189-1f28-451e-ad86-f1e7c420c40c","Type":"ContainerStarted","Data":"795048f67349ed2c156e3e6f938f8b6695a43f2e83749f297c53158a19c9d416"} Jul 01 16:58:21.367739 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.367692 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-ngvm6" podStartSLOduration=3.8447564610000002 podStartE2EDuration="23.367673626s" totalImagesPullingTime="19.522917165s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:21.367632568 +0000 UTC m=+24.607189244" watchObservedRunningTime="2026-07-01 16:58:21.367673626 +0000 UTC m=+24.607230296" Jul 01 16:58:21.368019 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:21.367987 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/konnectivity-agent-4pjmt" podStartSLOduration=3.8578711930000003 podStartE2EDuration="23.367980026s" totalImagesPullingTime="19.510108833s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:20.459329538 +0000 UTC m=+23.698886216" watchObservedRunningTime="2026-07-01 16:58:21.367980026 +0000 UTC m=+24.607536703" Jul 01 16:58:22.223732 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:22.223700 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:22.224041 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:22.223792 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:22.355160 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:22.355124 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" event={"ID":"7fcfbe51-d769-4162-89c5-95531c0af39e","Type":"ContainerStarted","Data":"ea96884727b70e29c318fd362982250f3b2f0df1d613919975e5b668bc66066a"} Jul 01 16:58:22.375637 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:22.375534 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-htpg4" podStartSLOduration=2.816294234 podStartE2EDuration="24.375519221s" totalImagesPullingTime="21.559224987s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:22.375325579 +0000 UTC m=+25.614882258" watchObservedRunningTime="2026-07-01 16:58:22.375519221 +0000 UTC m=+25.615075897" Jul 01 16:58:23.224199 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:23.223987 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:23.224424 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:23.223989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:23.224424 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:23.224291 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:23.224424 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:23.224370 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:23.360548 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:23.360512 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"9c81935e03b11f87e415e328da4a3a72f0949c08bd586357678489073a109417"} Jul 01 16:58:23.588150 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:23.588056 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:58:23.588897 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:23.588868 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:58:24.224206 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:24.224163 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:24.224384 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:24.224284 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:24.363372 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:24.363331 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:58:24.363933 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:24.363837 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-4pjmt" Jul 01 16:58:25.223823 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:25.223620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:25.224023 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:25.223620 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:25.224023 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:25.223917 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:25.224023 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:25.223989 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:25.368699 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:25.368671 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" event={"ID":"9ded3f35-ecd0-4c5f-8873-7ed0743a66d9","Type":"ContainerStarted","Data":"7cca86143e47bb8ed1b93509fdbd89cd593e417118c76a02c52ffa3fb4bd4148"} Jul 01 16:58:25.403240 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:25.403193 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" podStartSLOduration=7.056847396 podStartE2EDuration="27.403177924s" totalImagesPullingTime="20.346330528s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:25.402944973 +0000 UTC m=+28.642501649" watchObservedRunningTime="2026-07-01 16:58:25.403177924 +0000 UTC m=+28.642734601" Jul 01 16:58:26.224028 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.223989 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:26.224281 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:26.224096 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:26.372851 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.372792 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="2dec5d28ab88c90ff7e75cd305269b067e0dbd4550b399c824bd889800c8addd" exitCode=0 Jul 01 16:58:26.373314 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.372907 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"2dec5d28ab88c90ff7e75cd305269b067e0dbd4550b399c824bd889800c8addd"} Jul 01 16:58:26.374163 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.373578 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:26.374163 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.373640 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:26.374163 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.373680 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:26.389304 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.389275 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:26.389456 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:26.389446 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:27.191933 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.191900 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7tknz"] Jul 01 16:58:27.192152 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.192063 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:27.192225 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:27.192170 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:27.201499 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.201471 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vncqb"] Jul 01 16:58:27.201653 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.201589 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:27.201773 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:27.201732 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:27.202209 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.202177 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tz2kn"] Jul 01 16:58:27.202302 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:27.202285 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:27.202412 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:27.202388 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:28.378746 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:28.378653 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="adb3310eec37ccd61f576819f5b660d4ab45ab9a13543ec275b57949d21c4841" exitCode=0 Jul 01 16:58:28.379122 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:28.378739 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"adb3310eec37ccd61f576819f5b660d4ab45ab9a13543ec275b57949d21c4841"} Jul 01 16:58:29.223918 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:29.223681 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:29.224055 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:29.223688 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:29.224055 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:29.223975 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:29.224055 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:29.223716 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:29.224212 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:29.224073 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:29.224212 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:29.224148 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:30.385596 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:30.385558 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="c55361952533aed61d63f256c484974b36d05ea835f1b2a8dc0b54ccfc5674a2" exitCode=0 Jul 01 16:58:30.386163 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:30.385618 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"c55361952533aed61d63f256c484974b36d05ea835f1b2a8dc0b54ccfc5674a2"} Jul 01 16:58:30.927944 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:30.927907 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:30.928105 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:30.928060 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:30.928184 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:30.928132 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 16:59:02.928111395 +0000 UTC m=+66.167668059 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : object "openshift-multus"/"metrics-daemon-secret" not registered Jul 01 16:58:31.028947 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:31.028912 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:31.029106 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:31.028957 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:31.029106 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029065 2580 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:31.029175 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029074 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jul 01 16:58:31.029175 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029160 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret podName:ffc63e3c-606c-44ad-abd1-d8c725769df3 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:03.029144565 +0000 UTC m=+66.268701224 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret") pod "global-pull-secret-syncer-7tknz" (UID: "ffc63e3c-606c-44ad-abd1-d8c725769df3") : object "kube-system"/"original-pull-secret" not registered Jul 01 16:58:31.029175 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029166 2580 projected.go:291] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jul 01 16:58:31.029270 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029177 2580 projected.go:196] Error preparing data for projected volume kube-api-access-7f9hv for pod openshift-network-diagnostics/network-check-target-vncqb: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:31.029270 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.029216 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv podName:a4003675-8d98-49fd-8ea9-7d8633600257 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:03.029203077 +0000 UTC m=+66.268759732 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-7f9hv" (UniqueName: "kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv") pod "network-check-target-vncqb" (UID: "a4003675-8d98-49fd-8ea9-7d8633600257") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jul 01 16:58:31.224454 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:31.224420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:31.224454 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:31.224460 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:31.224697 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:31.224420 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:31.224697 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.224573 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-tz2kn" podUID="29b0fcf2-fa42-408f-9310-97306bbfb0de" Jul 01 16:58:31.224697 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.224640 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-vncqb" podUID="a4003675-8d98-49fd-8ea9-7d8633600257" Jul 01 16:58:31.224877 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:31.224711 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-7tknz" podUID="ffc63e3c-606c-44ad-abd1-d8c725769df3" Jul 01 16:58:32.989923 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:32.989832 2580 kubelet_node_status.go:695] "Recording event message for node" node="ip-10-0-138-241.ec2.internal" event="NodeReady" Jul 01 16:58:32.990524 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:32.990002 2580 kubelet_node_status.go:489] "Fast updating node status as it just became ready" Jul 01 16:58:33.033906 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.033862 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 16:58:33.036772 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.036749 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.039862 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.039792 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jul 01 16:58:33.039989 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.039968 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g89wz\"" Jul 01 16:58:33.040060 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.040047 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jul 01 16:58:33.041126 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.040981 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jul 01 16:58:33.047920 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.047893 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jul 01 16:58:33.048140 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.048118 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-p4jv9"] Jul 01 16:58:33.051043 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.051002 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.052829 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.052791 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 16:58:33.053773 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.053755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-shlnm\"" Jul 01 16:58:33.053874 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.053755 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jul 01 16:58:33.053874 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.053835 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jul 01 16:58:33.060967 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.060943 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-b8tlp"] Jul 01 16:58:33.062791 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.062758 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.064862 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.064841 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p4jv9"] Jul 01 16:58:33.065341 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.065317 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jul 01 16:58:33.066039 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.065603 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.066039 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.065673 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-9cnfg\"" Jul 01 16:58:33.066039 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.065892 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.077632 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.077611 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b8tlp"] Jul 01 16:58:33.143546 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143515 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.143785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143624 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.143785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143656 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.143785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143684 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xnmh6\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.143785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143748 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.143785 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143780 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.144048 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143863 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.144048 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.143899 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.228746 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.228695 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:58:33.228927 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.228831 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:58:33.228927 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.228854 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:58:33.231697 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.231657 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:58:33.232151 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.231982 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sw8bs\"" Jul 01 16:58:33.232151 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.232009 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:58:33.232151 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.232052 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:58:33.232151 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.232089 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:58:33.232151 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.232090 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8njwh\"" Jul 01 16:58:33.244388 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.244388 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244361 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.244557 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244423 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244690 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-tmp-dir\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244721 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.244833 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.244848 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.244911 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:33.744891119 +0000 UTC m=+36.984447777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244952 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.244977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-xnmh6\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-config-volume\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245028 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q88k7\" (UniqueName: \"kubernetes.io/projected/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-kube-api-access-q88k7\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245065 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245088 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-cert\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245113 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dsjkn\" (UniqueName: \"kubernetes.io/projected/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-kube-api-access-dsjkn\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245143 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.245517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245174 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.246270 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245483 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.246270 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245870 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.246270 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.245870 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.249302 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.249283 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.249408 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.249309 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.257831 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.257780 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xnmh6\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.258394 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.258374 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.285175 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.285138 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:33.346038 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346006 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-tmp-dir\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.346217 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346063 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-config-volume\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.346217 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346090 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-q88k7\" (UniqueName: \"kubernetes.io/projected/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-kube-api-access-q88k7\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.346217 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346164 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.346217 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346200 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-cert\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.346432 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346230 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-dsjkn\" (UniqueName: \"kubernetes.io/projected/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-kube-api-access-dsjkn\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.346432 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.346303 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:33.346432 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.346363 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:33.846344002 +0000 UTC m=+37.085900662 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:33.346570 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346429 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-tmp-dir\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.346796 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.346765 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-config-volume\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.349070 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.349045 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-cert\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.361136 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.361105 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q88k7\" (UniqueName: \"kubernetes.io/projected/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-kube-api-access-q88k7\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.362726 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.362693 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dsjkn\" (UniqueName: \"kubernetes.io/projected/fdcb703e-3e60-4b6f-a86a-9bf94608a44f-kube-api-access-dsjkn\") pod \"ingress-canary-b8tlp\" (UID: \"fdcb703e-3e60-4b6f-a86a-9bf94608a44f\") " pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.373323 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.373289 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-b8tlp" Jul 01 16:58:33.399100 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.399068 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:33.749175 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.749144 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:33.749345 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.749323 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:33.749400 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.749346 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:33.749435 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.749426 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:34.74940296 +0000 UTC m=+37.988959632 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:33.849911 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:33.849855 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:33.850068 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.849987 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:33.850068 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:33.850068 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:34.850045127 +0000 UTC m=+38.089601788 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:34.520999 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:34.520960 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-b8tlp"] Jul 01 16:58:34.524587 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:58:34.524554 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdcb703e_3e60_4b6f_a86a_9bf94608a44f.slice/crio-71ce060a7f485d80ae2fb91eed2187dbde6b6e155058f82433871b631327d698 WatchSource:0}: Error finding container 71ce060a7f485d80ae2fb91eed2187dbde6b6e155058f82433871b631327d698: Status 404 returned error can't find the container with id 71ce060a7f485d80ae2fb91eed2187dbde6b6e155058f82433871b631327d698 Jul 01 16:58:34.756486 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:34.756448 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:34.756660 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:34.756586 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:34.756660 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:34.756606 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:34.756660 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:34.756658 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:36.756643203 +0000 UTC m=+39.996199859 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:34.856911 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:34.856864 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:34.857098 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:34.857020 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:34.857098 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:34.857098 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:36.857078268 +0000 UTC m=+40.096634941 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:35.400512 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:35.400474 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="4ed1a3f4185a4562624921a1ff03a16b518c436ee2f590e6d411cbac797121de" exitCode=0 Jul 01 16:58:35.400702 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:35.400553 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"4ed1a3f4185a4562624921a1ff03a16b518c436ee2f590e6d411cbac797121de"} Jul 01 16:58:35.401847 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:35.401693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b8tlp" event={"ID":"fdcb703e-3e60-4b6f-a86a-9bf94608a44f","Type":"ContainerStarted","Data":"71ce060a7f485d80ae2fb91eed2187dbde6b6e155058f82433871b631327d698"} Jul 01 16:58:36.406682 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:36.406652 2580 generic.go:356] "Generic (PLEG): container finished" podID="c4909179-8cdd-41ef-b211-0915ff19bc87" containerID="19fa997e8aaa7836340706102c60de136c3e3185aaceb6ab15da2c9ecc8bc790" exitCode=0 Jul 01 16:58:36.407051 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:36.406693 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerDied","Data":"19fa997e8aaa7836340706102c60de136c3e3185aaceb6ab15da2c9ecc8bc790"} Jul 01 16:58:36.768910 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:36.768644 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:36.768910 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:36.768817 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:36.768910 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:36.768920 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:36.769178 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:36.768981 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:40.768962762 +0000 UTC m=+44.008519416 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:36.869889 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:36.869854 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:36.870090 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:36.870027 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:36.870164 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:36.870105 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:40.870082989 +0000 UTC m=+44.109639644 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:37.201094 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.201061 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:37.278455 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.278415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:37.389279 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.389244 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:37.412170 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.412142 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" event={"ID":"c4909179-8cdd-41ef-b211-0915ff19bc87","Type":"ContainerStarted","Data":"8fe001c3d145451b066ed3ae6274d11fb67a91787617c9f06bb1625d13108936"} Jul 01 16:58:37.439225 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.439161 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-k8sxt" podStartSLOduration=8.290950696 podStartE2EDuration="39.439124999s" totalImagesPullingTime="31.148174303s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=5 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:37.437935432 +0000 UTC m=+40.677492110" watchObservedRunningTime="2026-07-01 16:58:37.439124999 +0000 UTC m=+40.678681675" Jul 01 16:58:37.490174 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.490141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:37.589215 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.589181 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:37.689964 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:37.689926 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:38.178816 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:38.178757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:38.416382 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:38.416343 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-b8tlp" event={"ID":"fdcb703e-3e60-4b6f-a86a-9bf94608a44f","Type":"ContainerStarted","Data":"5f3ef7419c20081b933e6e675bf202b4294b7607276d76b9c39ec9e6a4277632"} Jul 01 16:58:38.438564 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:38.438466 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-b8tlp" podStartSLOduration=2.603636 podStartE2EDuration="5.438449485s" totalImagesPullingTime="2.834813485s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:58:38.438061477 +0000 UTC m=+41.677618155" watchObservedRunningTime="2026-07-01 16:58:38.438449485 +0000 UTC m=+41.678006162" Jul 01 16:58:38.545670 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:38.545632 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:38.611568 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:38.611532 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:39.021321 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:39.021282 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:39.124648 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:39.124616 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:40.799423 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:40.799384 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:40.799831 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:40.799563 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:40.799831 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:40.799581 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:40.799831 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:40.799637 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:48.799618592 +0000 UTC m=+52.039175262 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:40.900372 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:40.900340 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:40.900506 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:40.900461 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:40.900544 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:40.900510 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:58:48.90049609 +0000 UTC m=+52.140052746 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:42.172136 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:42.172099 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:48.857349 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:48.857307 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:58:48.857768 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:48.857475 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:58:48.857768 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:48.857493 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:58:48.857768 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:48.857547 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:04.857531235 +0000 UTC m=+68.097087890 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:58:48.958057 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:48.958021 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:58:48.958194 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:48.958175 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:58:48.958255 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:58:48.958245 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:04.958229336 +0000 UTC m=+68.197785991 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:58:50.219272 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:50.219234 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:51.955429 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:51.955387 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:52.055200 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:52.055163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:52.351319 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:52.351210 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:52.447128 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:52.447094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:52.547484 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:52.547449 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:53.318126 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:53.318098 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:53.421085 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:53.421049 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:53.632489 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:53.632404 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:54.315926 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:54.315893 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:54.402285 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:54.402247 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:54.533180 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:54.533141 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:54.609266 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:54.609188 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:56.294014 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:56.293974 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:58.396432 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:58.396403 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-g8mk4" Jul 01 16:58:58.634990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:58.634949 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:58.703185 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:58.703149 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:58:58.774782 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:58:58.774748 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:02.963653 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:02.963612 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 16:59:02.966725 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:02.966706 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jul 01 16:59:02.974025 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:02.974001 2580 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jul 01 16:59:02.974106 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:02.974097 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs podName:29b0fcf2-fa42-408f-9310-97306bbfb0de nodeName:}" failed. No retries permitted until 2026-07-01 17:00:06.974073561 +0000 UTC m=+130.213630218 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs") pod "network-metrics-daemon-tz2kn" (UID: "29b0fcf2-fa42-408f-9310-97306bbfb0de") : secret "metrics-daemon-secret" not found Jul 01 16:59:03.064425 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.064372 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:59:03.064584 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.064444 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:59:03.067361 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.067342 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jul 01 16:59:03.067467 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.067452 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jul 01 16:59:03.077242 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.077221 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jul 01 16:59:03.078513 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.078490 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ffc63e3c-606c-44ad-abd1-d8c725769df3-original-pull-secret\") pod \"global-pull-secret-syncer-7tknz\" (UID: \"ffc63e3c-606c-44ad-abd1-d8c725769df3\") " pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:59:03.087948 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.087922 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7f9hv\" (UniqueName: \"kubernetes.io/projected/a4003675-8d98-49fd-8ea9-7d8633600257-kube-api-access-7f9hv\") pod \"network-check-target-vncqb\" (UID: \"a4003675-8d98-49fd-8ea9-7d8633600257\") " pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:59:03.242211 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.242120 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-7tknz" Jul 01 16:59:03.252860 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.252828 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-sw8bs\"" Jul 01 16:59:03.259487 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.259459 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:59:03.411725 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.411694 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-7tknz"] Jul 01 16:59:03.414428 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:03.414394 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podffc63e3c_606c_44ad_abd1_d8c725769df3.slice/crio-288b7216254c85e80b6cac3135155813e4bfbf1792fee38132785b5e82f48ced WatchSource:0}: Error finding container 288b7216254c85e80b6cac3135155813e4bfbf1792fee38132785b5e82f48ced: Status 404 returned error can't find the container with id 288b7216254c85e80b6cac3135155813e4bfbf1792fee38132785b5e82f48ced Jul 01 16:59:03.423003 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.422976 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-vncqb"] Jul 01 16:59:03.426245 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:03.426218 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4003675_8d98_49fd_8ea9_7d8633600257.slice/crio-1db302e613fbef0f0028a67a193c73e98f17e5a4464e12375882debd5632faf0 WatchSource:0}: Error finding container 1db302e613fbef0f0028a67a193c73e98f17e5a4464e12375882debd5632faf0: Status 404 returned error can't find the container with id 1db302e613fbef0f0028a67a193c73e98f17e5a4464e12375882debd5632faf0 Jul 01 16:59:03.475116 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.475073 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7tknz" event={"ID":"ffc63e3c-606c-44ad-abd1-d8c725769df3","Type":"ContainerStarted","Data":"288b7216254c85e80b6cac3135155813e4bfbf1792fee38132785b5e82f48ced"} Jul 01 16:59:03.476110 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:03.476077 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vncqb" event={"ID":"a4003675-8d98-49fd-8ea9-7d8633600257","Type":"ContainerStarted","Data":"1db302e613fbef0f0028a67a193c73e98f17e5a4464e12375882debd5632faf0"} Jul 01 16:59:04.879509 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:04.879367 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:04.879509 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:04.879501 2580 projected.go:266] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jul 01 16:59:04.879509 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:04.879520 2580 projected.go:196] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-5f57596b68-99scq: secret "image-registry-tls" not found Jul 01 16:59:04.880222 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:04.879613 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls podName:29aca5e2-a134-4a66-9661-e00767645c69 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:36.87959173 +0000 UTC m=+100.119148403 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls") pod "image-registry-5f57596b68-99scq" (UID: "29aca5e2-a134-4a66-9661-e00767645c69") : secret "image-registry-tls" not found Jul 01 16:59:04.980821 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:04.980770 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:04.981018 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:04.980945 2580 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jul 01 16:59:04.981091 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:04.981022 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls podName:e6d11153-a3e0-4eb0-b679-3448bc2c4ab1 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:36.981002124 +0000 UTC m=+100.220558779 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls") pod "dns-default-p4jv9" (UID: "e6d11153-a3e0-4eb0-b679-3448bc2c4ab1") : secret "dns-default-metrics-tls" not found Jul 01 16:59:05.213840 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.213780 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:05.312109 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.312074 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-dcvzq"] Jul 01 16:59:05.326149 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.326072 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.332027 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.331997 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\"" Jul 01 16:59:05.332304 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.332271 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\"" Jul 01 16:59:05.332423 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.332278 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\"" Jul 01 16:59:05.332628 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.332609 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\"" Jul 01 16:59:05.333225 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.332920 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-vn9b8\"" Jul 01 16:59:05.350826 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.350755 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-dcvzq"] Jul 01 16:59:05.380974 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.380905 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\"" Jul 01 16:59:05.383344 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.383274 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/786e68e3-1a0e-4107-80c2-0b908527ac1f-serving-cert\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.384307 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.384109 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-trusted-ca\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.384307 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.384184 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-config\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.384307 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.384210 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qw46x\" (UniqueName: \"kubernetes.io/projected/786e68e3-1a0e-4107-80c2-0b908527ac1f-kube-api-access-qw46x\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.485245 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.484825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-config\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.485245 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.484870 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-qw46x\" (UniqueName: \"kubernetes.io/projected/786e68e3-1a0e-4107-80c2-0b908527ac1f-kube-api-access-qw46x\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.485245 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.484977 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/786e68e3-1a0e-4107-80c2-0b908527ac1f-serving-cert\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.485245 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.485049 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-trusted-ca\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.485872 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.485792 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-config\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.486123 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.486096 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/786e68e3-1a0e-4107-80c2-0b908527ac1f-trusted-ca\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.487849 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.487825 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/786e68e3-1a0e-4107-80c2-0b908527ac1f-serving-cert\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.508723 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.508659 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qw46x\" (UniqueName: \"kubernetes.io/projected/786e68e3-1a0e-4107-80c2-0b908527ac1f-kube-api-access-qw46x\") pod \"console-operator-7fdbdf9c99-dcvzq\" (UID: \"786e68e3-1a0e-4107-80c2-0b908527ac1f\") " pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.639016 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.638977 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:05.715703 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.715667 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:05.817768 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.817739 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-7fdbdf9c99-dcvzq"] Jul 01 16:59:05.818786 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:05.818754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:06.684487 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:06.684456 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod786e68e3_1a0e_4107_80c2_0b908527ac1f.slice/crio-5acb145ed229a0568be30bb0d55cb9ee3a5a881cf0da1fad0b9795ddc80474f8 WatchSource:0}: Error finding container 5acb145ed229a0568be30bb0d55cb9ee3a5a881cf0da1fad0b9795ddc80474f8: Status 404 returned error can't find the container with id 5acb145ed229a0568be30bb0d55cb9ee3a5a881cf0da1fad0b9795ddc80474f8 Jul 01 16:59:07.487975 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:07.487940 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" event={"ID":"786e68e3-1a0e-4107-80c2-0b908527ac1f","Type":"ContainerStarted","Data":"5acb145ed229a0568be30bb0d55cb9ee3a5a881cf0da1fad0b9795ddc80474f8"} Jul 01 16:59:08.492404 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:08.492340 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-7tknz" event={"ID":"ffc63e3c-606c-44ad-abd1-d8c725769df3","Type":"ContainerStarted","Data":"cc8b2e0decc15938e0bc4727b79d7742103b21357ed3f855193fa39b2241d4a4"} Jul 01 16:59:08.494108 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:08.494080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-vncqb" event={"ID":"a4003675-8d98-49fd-8ea9-7d8633600257","Type":"ContainerStarted","Data":"66c8486208b40349351952b0b234bb6f25aff663a543f99ccec553d86ad0aa4a"} Jul 01 16:59:08.494231 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:08.494212 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:59:08.510030 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:08.509956 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-7tknz" podStartSLOduration=65.853032907 podStartE2EDuration="1m10.509938774s" totalImagesPullingTime="4.656905867s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:08.509277502 +0000 UTC m=+71.748834181" watchObservedRunningTime="2026-07-01 16:59:08.509938774 +0000 UTC m=+71.749495451" Jul 01 16:59:08.525841 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:08.525771 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-vncqb" podStartSLOduration=65.883954344 podStartE2EDuration="1m10.525755163s" totalImagesPullingTime="4.641800819s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:08.525201702 +0000 UTC m=+71.764758381" watchObservedRunningTime="2026-07-01 16:59:08.525755163 +0000 UTC m=+71.765311842" Jul 01 16:59:10.500688 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.500661 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/0.log" Jul 01 16:59:10.501073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.500699 2580 generic.go:356] "Generic (PLEG): container finished" podID="786e68e3-1a0e-4107-80c2-0b908527ac1f" containerID="0e4b77564645d613b88323d2a24732ef0370b577bc33bbb859d4e74a98e0c332" exitCode=255 Jul 01 16:59:10.501073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.500746 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" event={"ID":"786e68e3-1a0e-4107-80c2-0b908527ac1f","Type":"ContainerDied","Data":"0e4b77564645d613b88323d2a24732ef0370b577bc33bbb859d4e74a98e0c332"} Jul 01 16:59:10.501073 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.500990 2580 scope.go:117] "RemoveContainer" containerID="0e4b77564645d613b88323d2a24732ef0370b577bc33bbb859d4e74a98e0c332" Jul 01 16:59:10.505686 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.505656 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:10.603312 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.603264 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:10.710456 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.710419 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:10.943416 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:10.943379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.061770 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.061733 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.124794 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.124756 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-lq86c"] Jul 01 16:59:11.128429 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.128401 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.134296 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.134268 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jul 01 16:59:11.134722 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.134667 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jul 01 16:59:11.135532 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.135511 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jul 01 16:59:11.135627 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.135506 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-zdcr6\"" Jul 01 16:59:11.135627 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.135513 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jul 01 16:59:11.143123 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.143099 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lq86c"] Jul 01 16:59:11.159251 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.159214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.231989 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.231905 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.231989 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.231938 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/513ed43d-ab12-4a4c-a9c7-bd939d193470-crio-socket\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.231989 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.231957 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.232225 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.232068 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/513ed43d-ab12-4a4c-a9c7-bd939d193470-data-volume\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.232225 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.232105 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4vbmn\" (UniqueName: \"kubernetes.io/projected/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-api-access-4vbmn\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.250669 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.250639 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.332793 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.332793 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332800 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/513ed43d-ab12-4a4c-a9c7-bd939d193470-crio-socket\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332846 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.332887 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332904 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/513ed43d-ab12-4a4c-a9c7-bd939d193470-data-volume\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332944 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4vbmn\" (UniqueName: \"kubernetes.io/projected/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-api-access-4vbmn\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.332962 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:11.832942509 +0000 UTC m=+75.072499182 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : configmap "kube-rbac-proxy" not found Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.332993 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/513ed43d-ab12-4a4c-a9c7-bd939d193470-crio-socket\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.333071 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.333014 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:11.333285 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.333076 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:11.833058254 +0000 UTC m=+75.072614939 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:11.333285 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.333258 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/513ed43d-ab12-4a4c-a9c7-bd939d193470-data-volume\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.343222 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.343194 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4vbmn\" (UniqueName: \"kubernetes.io/projected/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-api-access-4vbmn\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.504549 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.504474 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/1.log" Jul 01 16:59:11.504928 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.504854 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/0.log" Jul 01 16:59:11.504928 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.504888 2580 generic.go:356] "Generic (PLEG): container finished" podID="786e68e3-1a0e-4107-80c2-0b908527ac1f" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" exitCode=255 Jul 01 16:59:11.504998 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.504950 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" event={"ID":"786e68e3-1a0e-4107-80c2-0b908527ac1f","Type":"ContainerDied","Data":"3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28"} Jul 01 16:59:11.505037 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.505005 2580 scope.go:117] "RemoveContainer" containerID="0e4b77564645d613b88323d2a24732ef0370b577bc33bbb859d4e74a98e0c332" Jul 01 16:59:11.505213 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.505198 2580 scope.go:117] "RemoveContainer" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" Jul 01 16:59:11.505385 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.505367 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:11.630673 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.630638 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.733795 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.733754 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:11.836672 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.836583 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.836672 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.836625 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:11.836919 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.836716 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:11.836919 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.836773 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:11.836919 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.836781 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:12.836764483 +0000 UTC m=+76.076321138 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : configmap "kube-rbac-proxy" not found Jul 01 16:59:11.836919 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:11.836834 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:12.836818443 +0000 UTC m=+76.076375124 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:11.994987 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:11.994955 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:12.156172 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.156097 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:12.508993 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.508965 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/1.log" Jul 01 16:59:12.509382 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.509332 2580 scope.go:117] "RemoveContainer" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" Jul 01 16:59:12.509519 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:12.509499 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:12.565341 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.565307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:12.697779 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.697741 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:12.842771 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.842687 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:12.842771 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.842741 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:12.842930 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:12.842827 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:12.842930 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:12.842910 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:12.842998 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:12.842931 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:14.842916151 +0000 UTC m=+78.082472811 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : configmap "kube-rbac-proxy" not found Jul 01 16:59:12.842998 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:12.842960 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:14.842947374 +0000 UTC m=+78.082504030 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:12.859382 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.859350 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:12.987826 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:12.987772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:13.382305 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:13.382272 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:13.515435 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:13.515403 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:13.750820 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:13.750782 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hgs2r_f3e0f1c8-be07-4739-844d-7d02b51b42cc/dns-node-resolver/0.log" Jul 01 16:59:14.202081 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.202050 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-t44sn"] Jul 01 16:59:14.205633 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.205617 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.212451 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.212421 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\"" Jul 01 16:59:14.212611 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.212536 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-knzgh\"" Jul 01 16:59:14.213388 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.213368 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\"" Jul 01 16:59:14.219307 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.219289 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\"" Jul 01 16:59:14.219394 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.219323 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\"" Jul 01 16:59:14.328679 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.328641 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:14.355627 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.355586 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.355826 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.355648 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bf4h8\" (UniqueName: \"kubernetes.io/projected/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-kube-api-access-bf4h8\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.355826 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.355730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-key\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.404671 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.404636 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-t44sn"] Jul 01 16:59:14.456362 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.456274 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.456362 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.456342 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-bf4h8\" (UniqueName: \"kubernetes.io/projected/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-kube-api-access-bf4h8\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.456551 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.456366 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-key\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.456927 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.456904 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-cabundle\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.458814 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.458786 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-signing-key\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.467359 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.467330 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bf4h8\" (UniqueName: \"kubernetes.io/projected/8148b0f7-9b75-4bcb-a1be-bac9b78628bc-kube-api-access-bf4h8\") pod \"service-ca-7bb4f4bcc7-t44sn\" (UID: \"8148b0f7-9b75-4bcb-a1be-bac9b78628bc\") " pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.498004 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.497970 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:14.517516 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.517495 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" Jul 01 16:59:14.546080 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.546052 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-922gs_13204f90-24a2-4ee1-83c1-e6a31a5a76b8/node-ca/0.log" Jul 01 16:59:14.652174 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.652137 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7bb4f4bcc7-t44sn"] Jul 01 16:59:14.657882 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:14.657847 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8148b0f7_9b75_4bcb_a1be_bac9b78628bc.slice/crio-15102e9895785573b8e5942e9c89153f9a49a718e4c1ce52756f9cde28ada6f5 WatchSource:0}: Error finding container 15102e9895785573b8e5942e9c89153f9a49a718e4c1ce52756f9cde28ada6f5: Status 404 returned error can't find the container with id 15102e9895785573b8e5942e9c89153f9a49a718e4c1ce52756f9cde28ada6f5 Jul 01 16:59:14.859022 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.858926 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:14.859022 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:14.858967 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:14.859201 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:14.859072 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:14.859201 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:14.859103 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:14.859201 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:14.859157 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:18.859140842 +0000 UTC m=+82.098697497 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:14.859305 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:14.859204 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:18.859193088 +0000 UTC m=+82.098749744 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : configmap "kube-rbac-proxy" not found Jul 01 16:59:15.518499 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:15.518463 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" event={"ID":"8148b0f7-9b75-4bcb-a1be-bac9b78628bc","Type":"ContainerStarted","Data":"15102e9895785573b8e5942e9c89153f9a49a718e4c1ce52756f9cde28ada6f5"} Jul 01 16:59:15.548412 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:15.548374 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b8tlp_fdcb703e-3e60-4b6f-a86a-9bf94608a44f/serve-healthcheck-canary/0.log" Jul 01 16:59:15.639868 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:15.639834 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:15.640048 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:15.639899 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:15.640317 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:15.640302 2580 scope.go:117] "RemoveContainer" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" Jul 01 16:59:15.640497 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:15.640474 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:16.938835 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:16.938787 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:17.525233 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:17.525195 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" event={"ID":"8148b0f7-9b75-4bcb-a1be-bac9b78628bc","Type":"ContainerStarted","Data":"bcd9045e37691d0382a4d39eba25366c71b6584936f97d8f3c3ab004148f9ef5"} Jul 01 16:59:17.552366 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:17.552289 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7bb4f4bcc7-t44sn" podStartSLOduration=1.492084049 podStartE2EDuration="3.552272087s" totalImagesPullingTime="2.060188038s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:14 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:17.551345179 +0000 UTC m=+80.790901856" watchObservedRunningTime="2026-07-01 16:59:17.552272087 +0000 UTC m=+80.791828764" Jul 01 16:59:18.740839 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:18.740695 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:18.890220 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:18.890187 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:18.892099 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:18.892069 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:18.892257 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:18.892115 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:18.892341 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:18.892315 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:18.892449 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:18.892422 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:26.89240039 +0000 UTC m=+90.131957058 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:18.892890 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:18.892876 2580 configmap.go:193] Couldn't get configMap openshift-insights/kube-rbac-proxy: configmap "kube-rbac-proxy" not found Jul 01 16:59:18.892965 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:18.892954 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:26.892938612 +0000 UTC m=+90.132495270 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-rbac-proxy-cm" (UniqueName: "kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : configmap "kube-rbac-proxy" not found Jul 01 16:59:18.981876 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:18.981845 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:19.158470 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:19.158342 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:19.402716 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:19.402581 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:19.557781 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:19.557735 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:20.408711 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:20.408673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:20.813360 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:20.813323 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:21.043139 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:21.043105 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:21.229701 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:21.229666 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:21.454253 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:21.454220 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:22.200571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:22.200536 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:22.417748 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:22.417714 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:22.806673 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:22.806633 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:22.907571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:22.907538 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:23.045571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:23.045523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:23.223921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:23.223885 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:23.635325 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:23.635243 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:23.750027 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:23.749989 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:24.002491 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:24.002453 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:24.465557 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:24.465520 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:24.812047 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:24.811956 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:25.106688 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:25.106607 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:25.204461 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:25.204422 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:25.272633 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:25.272597 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:25.405516 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:25.405428 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:26.278127 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.278093 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:26.549598 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.549510 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:26.816383 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.816300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:26.960643 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.960610 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:26.960869 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.960649 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:26.960917 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:26.960874 2580 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jul 01 16:59:26.960967 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:26.960955 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls podName:513ed43d-ab12-4a4c-a9c7-bd939d193470 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:42.96093414 +0000 UTC m=+106.200490799 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls") pod "insights-runtime-extractor-lq86c" (UID: "513ed43d-ab12-4a4c-a9c7-bd939d193470") : secret "insights-runtime-extractor-tls" not found Jul 01 16:59:26.961145 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:26.961128 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/513ed43d-ab12-4a4c-a9c7-bd939d193470-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:27.021972 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:27.021930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:27.208508 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:27.208473 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:27.410251 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:27.410208 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:28.607960 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:28.607930 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:28.805683 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:28.805649 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:29.070183 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:29.070146 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:29.249429 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:29.249373 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:29.418532 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:29.418447 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:30.224398 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.224367 2580 scope.go:117] "RemoveContainer" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" Jul 01 16:59:30.560669 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.560595 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 16:59:30.561011 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.560996 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/1.log" Jul 01 16:59:30.561057 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.561032 2580 generic.go:356] "Generic (PLEG): container finished" podID="786e68e3-1a0e-4107-80c2-0b908527ac1f" containerID="5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45" exitCode=255 Jul 01 16:59:30.561102 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.561065 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" event={"ID":"786e68e3-1a0e-4107-80c2-0b908527ac1f","Type":"ContainerDied","Data":"5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45"} Jul 01 16:59:30.561146 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.561109 2580 scope.go:117] "RemoveContainer" containerID="3b4310cb0cecde716a683ef095bdaa93e10ac1a22fc3d6adf2a15fec92b3bc28" Jul 01 16:59:30.561469 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:30.561452 2580 scope.go:117] "RemoveContainer" containerID="5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45" Jul 01 16:59:30.561654 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:30.561636 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:31.565395 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:31.565368 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 16:59:32.539230 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:32.539199 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:35.640056 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:35.640019 2580 kubelet.go:2654] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:35.640461 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:35.640097 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:35.640461 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:35.640376 2580 scope.go:117] "RemoveContainer" containerID="5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45" Jul 01 16:59:35.640529 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:35.640508 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:36.941749 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:36.941691 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:36.944192 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:36.944170 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"image-registry-5f57596b68-99scq\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:36.952085 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:36.952060 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-g89wz\"" Jul 01 16:59:36.959279 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:36.959261 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:37.042575 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.042546 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:37.045054 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.045026 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/e6d11153-a3e0-4eb0-b679-3448bc2c4ab1-metrics-tls\") pod \"dns-default-p4jv9\" (UID: \"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1\") " pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:37.092338 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.092308 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 16:59:37.095398 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:37.095359 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29aca5e2_a134_4a66_9661_e00767645c69.slice/crio-79d8ebb9e7829cbf4b0468289c2d974bfddc8018a818bd805432e1b5e9dea269 WatchSource:0}: Error finding container 79d8ebb9e7829cbf4b0468289c2d974bfddc8018a818bd805432e1b5e9dea269: Status 404 returned error can't find the container with id 79d8ebb9e7829cbf4b0468289c2d974bfddc8018a818bd805432e1b5e9dea269 Jul 01 16:59:37.264681 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.264602 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-shlnm\"" Jul 01 16:59:37.272328 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.272301 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:37.413963 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.413933 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-p4jv9"] Jul 01 16:59:37.417121 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:37.417089 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode6d11153_a3e0_4eb0_b679_3448bc2c4ab1.slice/crio-6d9242a78b0d3d5174b32951d5b8afc0daa8805790167d5b15528bd9a417629e WatchSource:0}: Error finding container 6d9242a78b0d3d5174b32951d5b8afc0daa8805790167d5b15528bd9a417629e: Status 404 returned error can't find the container with id 6d9242a78b0d3d5174b32951d5b8afc0daa8805790167d5b15528bd9a417629e Jul 01 16:59:37.581243 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.581155 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4jv9" event={"ID":"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1","Type":"ContainerStarted","Data":"6d9242a78b0d3d5174b32951d5b8afc0daa8805790167d5b15528bd9a417629e"} Jul 01 16:59:37.582766 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.582735 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f57596b68-99scq" event={"ID":"29aca5e2-a134-4a66-9661-e00767645c69","Type":"ContainerStarted","Data":"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337"} Jul 01 16:59:37.582941 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.582786 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f57596b68-99scq" event={"ID":"29aca5e2-a134-4a66-9661-e00767645c69","Type":"ContainerStarted","Data":"79d8ebb9e7829cbf4b0468289c2d974bfddc8018a818bd805432e1b5e9dea269"} Jul 01 16:59:37.582941 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.582850 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:37.608589 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:37.608536 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5f57596b68-99scq" podStartSLOduration=99.608520153 podStartE2EDuration="1m39.608520153s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:37.607699785 +0000 UTC m=+100.847256463" watchObservedRunningTime="2026-07-01 16:59:37.608520153 +0000 UTC m=+100.848076831" Jul 01 16:59:39.450132 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:39.450094 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:39.499200 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:39.499169 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-vncqb" Jul 01 16:59:39.529916 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:39.529881 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:40.351443 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.351346 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7"] Jul 01 16:59:40.376035 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.376002 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-bf599"] Jul 01 16:59:40.376216 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.376142 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.381116 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.381088 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jul 01 16:59:40.381300 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.381124 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jul 01 16:59:40.381300 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.381152 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jul 01 16:59:40.381455 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.381409 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-2r45t\"" Jul 01 16:59:40.381455 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.381433 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jul 01 16:59:40.389502 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.389477 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7"] Jul 01 16:59:40.389609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.389515 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-bf599"] Jul 01 16:59:40.389609 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.389586 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" Jul 01 16:59:40.392568 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.392551 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"network-diagnostics-dockercfg-lsd5g\"" Jul 01 16:59:40.467997 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.467958 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pb64w\" (UniqueName: \"kubernetes.io/projected/ea4eeb2a-2526-4574-b520-edd19eae2e33-kube-api-access-pb64w\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.468388 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.468005 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5z8sj\" (UniqueName: \"kubernetes.io/projected/c58b2c02-41e8-4f2c-945b-24116210d6f0-kube-api-access-5z8sj\") pod \"network-check-source-56499946bc-bf599\" (UID: \"c58b2c02-41e8-4f2c-945b-24116210d6f0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" Jul 01 16:59:40.468388 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.468071 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea4eeb2a-2526-4574-b520-edd19eae2e33-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.569305 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.569252 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-5z8sj\" (UniqueName: \"kubernetes.io/projected/c58b2c02-41e8-4f2c-945b-24116210d6f0-kube-api-access-5z8sj\") pod \"network-check-source-56499946bc-bf599\" (UID: \"c58b2c02-41e8-4f2c-945b-24116210d6f0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" Jul 01 16:59:40.569305 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.569313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea4eeb2a-2526-4574-b520-edd19eae2e33-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.569597 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.569380 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-pb64w\" (UniqueName: \"kubernetes.io/projected/ea4eeb2a-2526-4574-b520-edd19eae2e33-kube-api-access-pb64w\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.571835 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.571795 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/ea4eeb2a-2526-4574-b520-edd19eae2e33-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.590203 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.590169 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5z8sj\" (UniqueName: \"kubernetes.io/projected/c58b2c02-41e8-4f2c-945b-24116210d6f0-kube-api-access-5z8sj\") pod \"network-check-source-56499946bc-bf599\" (UID: \"c58b2c02-41e8-4f2c-945b-24116210d6f0\") " pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" Jul 01 16:59:40.595085 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.595046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pb64w\" (UniqueName: \"kubernetes.io/projected/ea4eeb2a-2526-4574-b520-edd19eae2e33-kube-api-access-pb64w\") pod \"managed-serviceaccount-addon-agent-9568c8896-65cf7\" (UID: \"ea4eeb2a-2526-4574-b520-edd19eae2e33\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.595256 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.595080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4jv9" event={"ID":"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1","Type":"ContainerStarted","Data":"a3e862c84c7d500d5e7a0f3e9d5be712551dc04720a503bdd287bcb94848cafe"} Jul 01 16:59:40.595256 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.595134 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-p4jv9" event={"ID":"e6d11153-a3e0-4eb0-b679-3448bc2c4ab1","Type":"ContainerStarted","Data":"584e1e06a4169ca2c8ddbd8180cf809120cf84c01d6a1bdf6ca6fd7b2d0cf82d"} Jul 01 16:59:40.595256 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.595193 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:40.617900 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.617780 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:40.625433 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.625385 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-dns/dns-default-p4jv9" podStartSLOduration=65.303573642 podStartE2EDuration="1m7.625369644s" totalImagesPullingTime="2.321796002s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:58:33 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:40.625050121 +0000 UTC m=+103.864606799" watchObservedRunningTime="2026-07-01 16:59:40.625369644 +0000 UTC m=+103.864926299" Jul 01 16:59:40.694363 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.694322 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:40.697673 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.697654 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" Jul 01 16:59:40.703789 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.703769 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" Jul 01 16:59:40.819475 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.819437 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:40.851773 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.851740 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7"] Jul 01 16:59:40.854923 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:40.854900 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podea4eeb2a_2526_4574_b520_edd19eae2e33.slice/crio-d01440a5a2c7812fbf3bb234d0ceb29d7403183ea27bcb091068aacf2f5ebee3 WatchSource:0}: Error finding container d01440a5a2c7812fbf3bb234d0ceb29d7403183ea27bcb091068aacf2f5ebee3: Status 404 returned error can't find the container with id d01440a5a2c7812fbf3bb234d0ceb29d7403183ea27bcb091068aacf2f5ebee3 Jul 01 16:59:40.874005 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:40.873939 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-source-56499946bc-bf599"] Jul 01 16:59:40.876856 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:40.876834 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc58b2c02_41e8_4f2c_945b_24116210d6f0.slice/crio-c6837eb30854504ef1858315712f4762ea08fc802a7ef83e57b7d4c32e599840 WatchSource:0}: Error finding container c6837eb30854504ef1858315712f4762ea08fc802a7ef83e57b7d4c32e599840: Status 404 returned error can't find the container with id c6837eb30854504ef1858315712f4762ea08fc802a7ef83e57b7d4c32e599840 Jul 01 16:59:41.600218 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:41.600184 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" event={"ID":"c58b2c02-41e8-4f2c-945b-24116210d6f0","Type":"ContainerStarted","Data":"ff65f909a0b16dcfcc47ac70a92f5d7b785093de53120eb8d66f2e39df01efb5"} Jul 01 16:59:41.600721 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:41.600239 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" event={"ID":"c58b2c02-41e8-4f2c-945b-24116210d6f0","Type":"ContainerStarted","Data":"c6837eb30854504ef1858315712f4762ea08fc802a7ef83e57b7d4c32e599840"} Jul 01 16:59:41.601306 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:41.601281 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" event={"ID":"ea4eeb2a-2526-4574-b520-edd19eae2e33","Type":"ContainerStarted","Data":"d01440a5a2c7812fbf3bb234d0ceb29d7403183ea27bcb091068aacf2f5ebee3"} Jul 01 16:59:41.618224 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:41.618167 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-source-56499946bc-bf599" podStartSLOduration=1.618152229 podStartE2EDuration="1.618152229s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:40 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:41.617792877 +0000 UTC m=+104.857349566" watchObservedRunningTime="2026-07-01 16:59:41.618152229 +0000 UTC m=+104.857708883" Jul 01 16:59:42.988688 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:42.988635 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:42.992127 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:42.992092 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/513ed43d-ab12-4a4c-a9c7-bd939d193470-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-lq86c\" (UID: \"513ed43d-ab12-4a4c-a9c7-bd939d193470\") " pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:43.132994 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.132951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:43.237276 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.237242 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-lq86c" Jul 01 16:59:43.270345 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.270312 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:43.500413 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.500325 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:43.640754 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.640724 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:43.674906 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.674764 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-lq86c"] Jul 01 16:59:43.678474 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:43.678447 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod513ed43d_ab12_4a4c_a9c7_bd939d193470.slice/crio-bdd4d7dee0e165f3f8b204324768993a7d5ed86ee7b98eb3f3d3364324d0b018 WatchSource:0}: Error finding container bdd4d7dee0e165f3f8b204324768993a7d5ed86ee7b98eb3f3d3364324d0b018: Status 404 returned error can't find the container with id bdd4d7dee0e165f3f8b204324768993a7d5ed86ee7b98eb3f3d3364324d0b018 Jul 01 16:59:43.766671 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.766553 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:43.905547 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:43.905509 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:44.016047 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:44.016013 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:44.126393 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:44.126313 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:44.617316 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:44.617218 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq86c" event={"ID":"513ed43d-ab12-4a4c-a9c7-bd939d193470","Type":"ContainerStarted","Data":"bdd4d7dee0e165f3f8b204324768993a7d5ed86ee7b98eb3f3d3364324d0b018"} Jul 01 16:59:44.618751 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:44.618725 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" event={"ID":"ea4eeb2a-2526-4574-b520-edd19eae2e33","Type":"ContainerStarted","Data":"35c82c978848e9be3a2e2b0e76b3e41e28d705020263de7c3b50401c963647ce"} Jul 01 16:59:44.654550 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:44.654490 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-9568c8896-65cf7" podStartSLOduration=1.93334123 podStartE2EDuration="4.654466305s" totalImagesPullingTime="2.721125075s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:44.6543427 +0000 UTC m=+107.893899378" watchObservedRunningTime="2026-07-01 16:59:44.654466305 +0000 UTC m=+107.894022986" Jul 01 16:59:45.699182 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:45.699140 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:47.631046 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:47.630940 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq86c" event={"ID":"513ed43d-ab12-4a4c-a9c7-bd939d193470","Type":"ContainerStarted","Data":"2e3d8fe50861894a06abc8edab515828bc9f5e0a2c84601588e51183d594a3fa"} Jul 01 16:59:48.224422 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.224395 2580 scope.go:117] "RemoveContainer" containerID="5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45" Jul 01 16:59:48.224585 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:48.224559 2580 pod_workers.go:1324] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-7fdbdf9c99-dcvzq_openshift-console-operator(786e68e3-1a0e-4107-80c2-0b908527ac1f)\"" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podUID="786e68e3-1a0e-4107-80c2-0b908527ac1f" Jul 01 16:59:48.500755 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.500660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:48.556704 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.556667 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-gxdxh"] Jul 01 16:59:48.560042 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.560016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.563409 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.563382 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jul 01 16:59:48.563775 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.563755 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jul 01 16:59:48.564424 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.564404 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jul 01 16:59:48.564512 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.564410 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jul 01 16:59:48.564757 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.564737 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jul 01 16:59:48.564936 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.564821 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-p48bm\"" Jul 01 16:59:48.571413 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.571388 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-gxdxh"] Jul 01 16:59:48.598517 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.598470 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:48.635929 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.635902 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4869407-5981-463b-9f9e-5ca9599b3337-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.636373 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.635946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.636373 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.636003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vszcg\" (UniqueName: \"kubernetes.io/projected/f4869407-5981-463b-9f9e-5ca9599b3337-kube-api-access-vszcg\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.636373 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.636044 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.637102 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.637076 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq86c" event={"ID":"513ed43d-ab12-4a4c-a9c7-bd939d193470","Type":"ContainerStarted","Data":"7942d77b19dad2f3ae89e2d8a9fbea63050de883398d8e84f4137d7676858c47"} Jul 01 16:59:48.737348 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.737308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-vszcg\" (UniqueName: \"kubernetes.io/projected/f4869407-5981-463b-9f9e-5ca9599b3337-kube-api-access-vszcg\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.737535 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.737375 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.737608 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.737560 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4869407-5981-463b-9f9e-5ca9599b3337-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.737662 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.737605 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.738375 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.738351 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f4869407-5981-463b-9f9e-5ca9599b3337-metrics-client-ca\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.740175 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.740149 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-tls\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.740282 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.740222 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f4869407-5981-463b-9f9e-5ca9599b3337-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.747908 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.747882 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vszcg\" (UniqueName: \"kubernetes.io/projected/f4869407-5981-463b-9f9e-5ca9599b3337-kube-api-access-vszcg\") pod \"prometheus-operator-7b7d896497-gxdxh\" (UID: \"f4869407-5981-463b-9f9e-5ca9599b3337\") " pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:48.871972 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:48.871888 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" Jul 01 16:59:49.040477 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:49.040442 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-7b7d896497-gxdxh"] Jul 01 16:59:49.044922 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:49.044892 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf4869407_5981_463b_9f9e_5ca9599b3337.slice/crio-d2b9d1da8a5edc6f6f8ae33d23e70f058cddecf97042daee0427f75a73d2a531 WatchSource:0}: Error finding container d2b9d1da8a5edc6f6f8ae33d23e70f058cddecf97042daee0427f75a73d2a531: Status 404 returned error can't find the container with id d2b9d1da8a5edc6f6f8ae33d23e70f058cddecf97042daee0427f75a73d2a531 Jul 01 16:59:49.642654 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:49.642612 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" event={"ID":"f4869407-5981-463b-9f9e-5ca9599b3337","Type":"ContainerStarted","Data":"d2b9d1da8a5edc6f6f8ae33d23e70f058cddecf97042daee0427f75a73d2a531"} Jul 01 16:59:50.604200 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:50.604160 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-p4jv9" Jul 01 16:59:50.649481 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:50.649438 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-lq86c" event={"ID":"513ed43d-ab12-4a4c-a9c7-bd939d193470","Type":"ContainerStarted","Data":"db7ae41510ac1cbabf58d164ec82a5449fafdd2dad1caa106d8bfa5e7ed27d1a"} Jul 01 16:59:50.712754 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:50.712693 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-lq86c" podStartSLOduration=33.031126177 podStartE2EDuration="39.712676116s" totalImagesPullingTime="6.681549939s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 16:59:11 +0000 UTC" imagePullSessionsCount=3 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:50.712514758 +0000 UTC m=+113.952071434" watchObservedRunningTime="2026-07-01 16:59:50.712676116 +0000 UTC m=+113.952232792" Jul 01 16:59:51.656595 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:51.656437 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" event={"ID":"f4869407-5981-463b-9f9e-5ca9599b3337","Type":"ContainerStarted","Data":"2f4830308569dffe7bb845b172b222f2ea7aea0ef11bfc8e4fc0998cdbd97936"} Jul 01 16:59:51.656595 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:51.656493 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" event={"ID":"f4869407-5981-463b-9f9e-5ca9599b3337","Type":"ContainerStarted","Data":"2a9d42a9de1e0f693a4936c5ed95cee44bc44f7fbffaeccdbc000139a0ce7692"} Jul 01 16:59:51.709718 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:51.709658 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-7b7d896497-gxdxh" podStartSLOduration=1.356595906 podStartE2EDuration="3.709639724s" totalImagesPullingTime="2.353043818s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:48 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:51.708214311 +0000 UTC m=+114.947770989" watchObservedRunningTime="2026-07-01 16:59:51.709639724 +0000 UTC m=+114.949196401" Jul 01 16:59:51.767718 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:51.767679 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:53.788251 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:53.788212 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:54.288697 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.288655 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-lnrxx"] Jul 01 16:59:54.292347 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.292327 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.302036 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.302001 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jul 01 16:59:54.302672 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.302654 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jul 01 16:59:54.302840 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.302822 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-ttjh4\"" Jul 01 16:59:54.302988 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.302973 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jul 01 16:59:54.386358 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386318 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386542 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386387 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4qsml\" (UniqueName: \"kubernetes.io/projected/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-kube-api-access-4qsml\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386542 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-sys\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386542 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386457 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386542 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386511 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-textfile\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386689 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386564 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-wtmp\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386689 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386593 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-metrics-client-ca\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386689 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386615 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.386689 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.386643 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-root\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487294 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-metrics-client-ca\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487294 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487298 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487322 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-root\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487370 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487393 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4qsml\" (UniqueName: \"kubernetes.io/projected/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-kube-api-access-4qsml\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487419 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-sys\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487442 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487467 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-textfile\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487467 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-root\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487538 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-wtmp\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487935 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:54.487574 2580 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jul 01 16:59:54.487935 ip-10-0-138-241 kubenswrapper[2580]: E0701 16:59:54.487633 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls podName:fc1ca859-c7ec-48fe-81ee-107c9a04bf55 nodeName:}" failed. No retries permitted until 2026-07-01 16:59:54.987612981 +0000 UTC m=+118.227169642 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls") pod "node-exporter-lnrxx" (UID: "fc1ca859-c7ec-48fe-81ee-107c9a04bf55") : secret "node-exporter-tls" not found Jul 01 16:59:54.487935 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487647 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-wtmp\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487935 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487758 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-sys\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.487935 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487924 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-textfile\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.488119 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.487964 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-accelerators-collector-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.488119 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.488027 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-metrics-client-ca\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.489765 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.489744 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.504043 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.504018 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4qsml\" (UniqueName: \"kubernetes.io/projected/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-kube-api-access-4qsml\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.749458 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.749420 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:54.886961 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.886922 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:54.990988 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.990945 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:54.993265 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:54.993242 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/fc1ca859-c7ec-48fe-81ee-107c9a04bf55-node-exporter-tls\") pod \"node-exporter-lnrxx\" (UID: \"fc1ca859-c7ec-48fe-81ee-107c9a04bf55\") " pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:55.180926 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.180835 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:55.201929 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.201900 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-lnrxx" Jul 01 16:59:55.213617 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:55.213576 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfc1ca859_c7ec_48fe_81ee_107c9a04bf55.slice/crio-1c80ae6a115e16221ca60ae00bfcd3117cd04821912abdff975dbde9a56184c4 WatchSource:0}: Error finding container 1c80ae6a115e16221ca60ae00bfcd3117cd04821912abdff975dbde9a56184c4: Status 404 returned error can't find the container with id 1c80ae6a115e16221ca60ae00bfcd3117cd04821912abdff975dbde9a56184c4 Jul 01 16:59:55.288043 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.288006 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:55.428971 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.428935 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:55.675345 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.675289 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnrxx" event={"ID":"fc1ca859-c7ec-48fe-81ee-107c9a04bf55","Type":"ContainerStarted","Data":"1c80ae6a115e16221ca60ae00bfcd3117cd04821912abdff975dbde9a56184c4"} Jul 01 16:59:55.995526 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:55.995480 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:56.204260 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:56.204219 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:56.680584 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:56.680547 2580 generic.go:356] "Generic (PLEG): container finished" podID="fc1ca859-c7ec-48fe-81ee-107c9a04bf55" containerID="78421dbe1a8eae58bc0f11699190f4ab3e6b5d0c58fef791fc338253e12ea1bd" exitCode=0 Jul 01 16:59:56.680740 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:56.680609 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnrxx" event={"ID":"fc1ca859-c7ec-48fe-81ee-107c9a04bf55","Type":"ContainerDied","Data":"78421dbe1a8eae58bc0f11699190f4ab3e6b5d0c58fef791fc338253e12ea1bd"} Jul 01 16:59:56.963842 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:56.963780 2580 patch_prober.go:28] interesting pod/image-registry-5f57596b68-99scq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]} Jul 01 16:59:56.963983 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:56.963874 2580 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-5f57596b68-99scq" podUID="29aca5e2-a134-4a66-9661-e00767645c69" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 16:59:57.687410 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:57.687378 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnrxx" event={"ID":"fc1ca859-c7ec-48fe-81ee-107c9a04bf55","Type":"ContainerStarted","Data":"4f7ba439e67b94f84a911b6d78307a5b634577a9fd4db692d253b68ec78247a1"} Jul 01 16:59:57.687826 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:57.687457 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-lnrxx" event={"ID":"fc1ca859-c7ec-48fe-81ee-107c9a04bf55","Type":"ContainerStarted","Data":"26f14d6d3a3fe7890581c11abc46019f047042ea9fb8f4e0351fc07c57b3de24"} Jul 01 16:59:57.736043 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:57.735980 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-lnrxx" podStartSLOduration=2.783527181 podStartE2EDuration="3.735964241s" totalImagesPullingTime="886.949569ms" totalInitContainerRuntime="65.487491ms" isStatefulPod=true podCreationTimestamp="2026-07-01 16:59:54 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:57.735457596 +0000 UTC m=+120.975014273" watchObservedRunningTime="2026-07-01 16:59:57.735964241 +0000 UTC m=+120.975520917" Jul 01 16:59:57.785873 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:57.785830 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:57.971690 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:57.971658 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:58.063078 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.063041 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:58.477439 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.477405 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-69995699bc-tqnfz"] Jul 01 16:59:58.481061 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.481040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.491143 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.491118 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\"" Jul 01 16:59:58.491269 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.491175 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\"" Jul 01 16:59:58.491269 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.491176 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\"" Jul 01 16:59:58.491960 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.491939 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\"" Jul 01 16:59:58.492082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.491985 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\"" Jul 01 16:59:58.492082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.492005 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-b4bd2fjfc2s6i\"" Jul 01 16:59:58.492082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.492064 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-8gtdq\"" Jul 01 16:59:58.522428 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.522398 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69995699bc-tqnfz"] Jul 01 16:59:58.592906 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.592875 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 16:59:58.620646 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620604 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620646 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620650 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620677 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620728 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-grpc-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620830 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620873 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxnth\" (UniqueName: \"kubernetes.io/projected/db99495f-b0ab-4c67-a198-87461a7f7bb4-kube-api-access-wxnth\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.620921 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620900 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db99495f-b0ab-4c67-a198-87461a7f7bb4-metrics-client-ca\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.621078 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.620939 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722061 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722022 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722092 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-wxnth\" (UniqueName: \"kubernetes.io/projected/db99495f-b0ab-4c67-a198-87461a7f7bb4-kube-api-access-wxnth\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722132 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db99495f-b0ab-4c67-a198-87461a7f7bb4-metrics-client-ca\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722162 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722286 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722313 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.722571 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.722494 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-grpc-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.723067 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.723038 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/db99495f-b0ab-4c67-a198-87461a7f7bb4-metrics-client-ca\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725037 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725015 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-grpc-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725154 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725134 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725195 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725136 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725309 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725285 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725401 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725381 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.725735 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.725717 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/db99495f-b0ab-4c67-a198-87461a7f7bb4-secret-thanos-querier-tls\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.742584 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.742512 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxnth\" (UniqueName: \"kubernetes.io/projected/db99495f-b0ab-4c67-a198-87461a7f7bb4-kube-api-access-wxnth\") pod \"thanos-querier-69995699bc-tqnfz\" (UID: \"db99495f-b0ab-4c67-a198-87461a7f7bb4\") " pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.789680 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.789647 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 16:59:58.826498 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.826465 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:58.928162 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.928124 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-65448c9fc7-dtrb2"] Jul 01 16:59:58.932241 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.932218 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:58.939028 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.938969 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jul 01 16:59:58.939210 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.939188 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-1er1s6d5672e0\"" Jul 01 16:59:58.939971 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.939956 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jul 01 16:59:58.947285 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.947264 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jul 01 16:59:58.947493 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.947480 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jul 01 16:59:58.947565 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.947552 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-5bnpz\"" Jul 01 16:59:58.959268 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.959241 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65448c9fc7-dtrb2"] Jul 01 16:59:58.988239 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:58.988212 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-69995699bc-tqnfz"] Jul 01 16:59:58.989511 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:58.989489 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb99495f_b0ab_4c67_a198_87461a7f7bb4.slice/crio-99d9f5dc41545825e5e80f4bd7b099661e24fd0923f89e1c9f0ebaee42773689 WatchSource:0}: Error finding container 99d9f5dc41545825e5e80f4bd7b099661e24fd0923f89e1c9f0ebaee42773689: Status 404 returned error can't find the container with id 99d9f5dc41545825e5e80f4bd7b099661e24fd0923f89e1c9f0ebaee42773689 Jul 01 16:59:59.015717 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.015683 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:59.024971 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.024950 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-audit-log\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025070 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025003 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-client-certs\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025070 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025049 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-metrics-server-audit-profiles\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025141 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025077 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-tls\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025141 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025110 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-client-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025141 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025138 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.025261 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.025167 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7xbxc\" (UniqueName: \"kubernetes.io/projected/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-kube-api-access-7xbxc\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.106559 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.106523 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 16:59:59.125707 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125676 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-7xbxc\" (UniqueName: \"kubernetes.io/projected/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-kube-api-access-7xbxc\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.125819 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125720 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-audit-log\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.125819 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125763 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-client-certs\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.125932 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125825 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-metrics-server-audit-profiles\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.125932 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125868 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-tls\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.125932 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125901 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-client-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.126082 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.125943 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.126236 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.126179 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-audit-log\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.126680 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.126651 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.126896 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.126871 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-metrics-server-audit-profiles\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.128570 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.128545 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-client-ca-bundle\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.128990 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.128971 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-tls\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.129037 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.128971 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-secret-metrics-server-client-certs\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.156668 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.156599 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7xbxc\" (UniqueName: \"kubernetes.io/projected/4676fb6a-72b0-4cb2-b053-133b2c4ded9c-kube-api-access-7xbxc\") pod \"metrics-server-65448c9fc7-dtrb2\" (UID: \"4676fb6a-72b0-4cb2-b053-133b2c4ded9c\") " pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.224322 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.224286 2580 scope.go:117] "RemoveContainer" containerID="5e8a1d8797d03673f4baeb3dd14ccc755090c7b7d1a95b03f37161ec82658d45" Jul 01 16:59:59.242156 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.242133 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 16:59:59.390063 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.390041 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-65448c9fc7-dtrb2"] Jul 01 16:59:59.392016 ip-10-0-138-241 kubenswrapper[2580]: W0701 16:59:59.391992 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4676fb6a_72b0_4cb2_b053_133b2c4ded9c.slice/crio-f928b2cf4699f1f1e359b575410958d869c59d721a3de465ef5f0b7bdc888770 WatchSource:0}: Error finding container f928b2cf4699f1f1e359b575410958d869c59d721a3de465ef5f0b7bdc888770: Status 404 returned error can't find the container with id f928b2cf4699f1f1e359b575410958d869c59d721a3de465ef5f0b7bdc888770 Jul 01 16:59:59.696344 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.696286 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" event={"ID":"4676fb6a-72b0-4cb2-b053-133b2c4ded9c","Type":"ContainerStarted","Data":"f928b2cf4699f1f1e359b575410958d869c59d721a3de465ef5f0b7bdc888770"} Jul 01 16:59:59.698426 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.698392 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 16:59:59.698578 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.698480 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" event={"ID":"786e68e3-1a0e-4107-80c2-0b908527ac1f","Type":"ContainerStarted","Data":"cf054544f07ec5f194540bc18a9293577242252200848e13a569556ae7dd4a10"} Jul 01 16:59:59.698962 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.698936 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 16:59:59.699904 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.699877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"99d9f5dc41545825e5e80f4bd7b099661e24fd0923f89e1c9f0ebaee42773689"} Jul 01 16:59:59.730008 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.729951 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" podStartSLOduration=51.810919482 podStartE2EDuration="54.729933916s" totalImagesPullingTime="2.919014434s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:05 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 16:59:59.72775649 +0000 UTC m=+122.967313168" watchObservedRunningTime="2026-07-01 16:59:59.729933916 +0000 UTC m=+122.969490592" Jul 01 16:59:59.843714 ip-10-0-138-241 kubenswrapper[2580]: I0701 16:59:59.843683 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-7fdbdf9c99-dcvzq" Jul 01 17:00:00.015477 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.015369 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:00.151933 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.151897 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-5499b6fb8f-9sghz"] Jul 01 17:00:00.155417 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.155389 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:00.161037 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.161011 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jul 01 17:00:00.161180 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.161080 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-kkd5c\"" Jul 01 17:00:00.161180 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.161123 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jul 01 17:00:00.167186 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.166791 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:00.174024 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.173890 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-9sghz"] Jul 01 17:00:00.238506 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.238462 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-svgcm\" (UniqueName: \"kubernetes.io/projected/0f1d0bc2-7b05-4877-82e5-e176ebe68fc9-kube-api-access-svgcm\") pod \"downloads-5499b6fb8f-9sghz\" (UID: \"0f1d0bc2-7b05-4877-82e5-e176ebe68fc9\") " pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:00.270824 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.270694 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:00.339670 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.339633 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-svgcm\" (UniqueName: \"kubernetes.io/projected/0f1d0bc2-7b05-4877-82e5-e176ebe68fc9-kube-api-access-svgcm\") pod \"downloads-5499b6fb8f-9sghz\" (UID: \"0f1d0bc2-7b05-4877-82e5-e176ebe68fc9\") " pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:00.357747 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.357710 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-svgcm\" (UniqueName: \"kubernetes.io/projected/0f1d0bc2-7b05-4877-82e5-e176ebe68fc9-kube-api-access-svgcm\") pod \"downloads-5499b6fb8f-9sghz\" (UID: \"0f1d0bc2-7b05-4877-82e5-e176ebe68fc9\") " pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:00.378860 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.378696 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:00.468678 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:00.468632 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:01.004949 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.004914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:01.210386 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.210349 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:01.444723 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.444693 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:01.631759 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.631725 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-5499b6fb8f-9sghz"] Jul 01 17:00:01.637654 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:00:01.637613 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0f1d0bc2_7b05_4877_82e5_e176ebe68fc9.slice/crio-15e4ad798d68cb6c769fa870aa387206f1f21692e079d59e53013244541ae974 WatchSource:0}: Error finding container 15e4ad798d68cb6c769fa870aa387206f1f21692e079d59e53013244541ae974: Status 404 returned error can't find the container with id 15e4ad798d68cb6c769fa870aa387206f1f21692e079d59e53013244541ae974 Jul 01 17:00:01.710534 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.710499 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"b11089b8d2fd99f4bc8413f5f3599d75d0716fbe620ccc58f93fb52013eacb3d"} Jul 01 17:00:01.710659 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.710589 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"79cd39268c34443cf9e11a326794d6f44dde47c02958bfd5740e8d005fa30f74"} Jul 01 17:00:01.712359 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.712327 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" event={"ID":"4676fb6a-72b0-4cb2-b053-133b2c4ded9c","Type":"ContainerStarted","Data":"f30b4fb5f1336094e1d38e46576d6b96be33e59f680bbaa16c1a53e5dde1d794"} Jul 01 17:00:01.717232 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.716456 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-9sghz" event={"ID":"0f1d0bc2-7b05-4877-82e5-e176ebe68fc9","Type":"ContainerStarted","Data":"15e4ad798d68cb6c769fa870aa387206f1f21692e079d59e53013244541ae974"} Jul 01 17:00:01.780121 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.780062 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" podStartSLOduration=1.712125571 podStartE2EDuration="3.780044395s" totalImagesPullingTime="2.067918824s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:01.778148292 +0000 UTC m=+125.017704969" watchObservedRunningTime="2026-07-01 17:00:01.780044395 +0000 UTC m=+125.019601071" Jul 01 17:00:01.908468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:01.908432 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:02.353336 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.353298 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:02.416485 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.416417 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 17:00:02.540696 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.540655 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:02.724757 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.724728 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"f81aa2d4b8e6f13d0e68b2b2ece707bfdd24a41cce9a13475d9186aa157da066"} Jul 01 17:00:02.753718 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.753673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:02.881820 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:02.881770 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:03.017504 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.017396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:03.732691 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.732655 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"409e4d356c88d08aecd774c8552e91116c0512cc484a42616eac9ab59bdf5a3d"} Jul 01 17:00:03.733152 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.732716 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"d987bb931cbae07087f73b77edd01a736f83588665625c830a133c14c25ea213"} Jul 01 17:00:03.733152 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.732743 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" event={"ID":"db99495f-b0ab-4c67-a198-87461a7f7bb4","Type":"ContainerStarted","Data":"6f791475f312d4810eeabbf380719fd79481b885864d418dd37f2938b9471d44"} Jul 01 17:00:03.733152 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.732858 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 17:00:03.774368 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:03.774304 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" podStartSLOduration=1.941944211 podStartE2EDuration="5.774287097s" totalImagesPullingTime="3.832342886s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:59:58 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:03.773230138 +0000 UTC m=+127.012786815" watchObservedRunningTime="2026-07-01 17:00:03.774287097 +0000 UTC m=+127.013843774" Jul 01 17:00:07.002751 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.002709 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 17:00:07.005373 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.005348 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/29b0fcf2-fa42-408f-9310-97306bbfb0de-metrics-certs\") pod \"network-metrics-daemon-tz2kn\" (UID: \"29b0fcf2-fa42-408f-9310-97306bbfb0de\") " pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 17:00:07.174403 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.170907 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-8njwh\"" Jul 01 17:00:07.178101 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.178081 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-tz2kn" Jul 01 17:00:07.337341 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.337299 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-tz2kn"] Jul 01 17:00:07.341464 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:00:07.341432 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29b0fcf2_fa42_408f_9310_97306bbfb0de.slice/crio-c4a88bf3fc4d7ee51fb026d8f3242856f14210eced9d0e2205ec2d1d12bdbb85 WatchSource:0}: Error finding container c4a88bf3fc4d7ee51fb026d8f3242856f14210eced9d0e2205ec2d1d12bdbb85: Status 404 returned error can't find the container with id c4a88bf3fc4d7ee51fb026d8f3242856f14210eced9d0e2205ec2d1d12bdbb85 Jul 01 17:00:07.578722 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.578595 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:07.751258 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.751219 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz2kn" event={"ID":"29b0fcf2-fa42-408f-9310-97306bbfb0de","Type":"ContainerStarted","Data":"c4a88bf3fc4d7ee51fb026d8f3242856f14210eced9d0e2205ec2d1d12bdbb85"} Jul 01 17:00:07.876669 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:07.876487 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:08.424251 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.424208 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:08.429370 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.429341 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.435167 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435142 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jul 01 17:00:08.435341 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435173 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jul 01 17:00:08.435921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435604 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jul 01 17:00:08.435921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435661 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-fw6hw\"" Jul 01 17:00:08.435921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435723 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jul 01 17:00:08.435921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.435747 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jul 01 17:00:08.445924 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.445902 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:08.515460 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515427 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.515638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515481 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.515638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515520 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.515638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515541 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.515638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515572 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gkng6\" (UniqueName: \"kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.515789 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.515703 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616109 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616068 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616119 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616161 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-gkng6\" (UniqueName: \"kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616233 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616259 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.616508 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.616308 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.617064 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.617036 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.617271 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.617247 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.617638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.617612 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.619614 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.619592 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.620154 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.620133 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.632511 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.632488 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gkng6\" (UniqueName: \"kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6\") pod \"console-77dc8866db-jtpsl\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.742134 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.742086 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:08.757221 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.757182 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz2kn" event={"ID":"29b0fcf2-fa42-408f-9310-97306bbfb0de","Type":"ContainerStarted","Data":"f5aa554395d53101a5b51902d5eda01aa6474a279e7ab6e1d7df0bd40beb9cb3"} Jul 01 17:00:08.918080 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:08.918044 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:08.921578 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:00:08.921550 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod23143c9e_a088_4bd7_bf69_e120cf604757.slice/crio-939ed38cb7478d2c1b2b48efc837a4ef03b42957298a74f7cafcf1afd0789a29 WatchSource:0}: Error finding container 939ed38cb7478d2c1b2b48efc837a4ef03b42957298a74f7cafcf1afd0789a29: Status 404 returned error can't find the container with id 939ed38cb7478d2c1b2b48efc837a4ef03b42957298a74f7cafcf1afd0789a29 Jul 01 17:00:09.266055 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.266014 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:09.358323 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.358286 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:09.745298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.745265 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-69995699bc-tqnfz" Jul 01 17:00:09.764436 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.764398 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-tz2kn" event={"ID":"29b0fcf2-fa42-408f-9310-97306bbfb0de","Type":"ContainerStarted","Data":"357c4bb753898d29651ced04c056080baf641be40b45a3096263916096ad71d1"} Jul 01 17:00:09.766257 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.766226 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dc8866db-jtpsl" event={"ID":"23143c9e-a088-4bd7-bf69-e120cf604757","Type":"ContainerStarted","Data":"939ed38cb7478d2c1b2b48efc837a4ef03b42957298a74f7cafcf1afd0789a29"} Jul 01 17:00:09.767927 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.767889 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:09.894310 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.894263 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:09.996768 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:09.996678 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:10.458701 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:10.458660 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:13.671373 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:13.671311 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:13.837446 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:13.837407 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:16.090718 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.090661 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-tz2kn" podStartSLOduration=136.865857225 podStartE2EDuration="2m18.090643895s" totalImagesPullingTime="1.22478667s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 16:57:58 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:09.813016762 +0000 UTC m=+133.052573440" watchObservedRunningTime="2026-07-01 17:00:16.090643895 +0000 UTC m=+139.330200572" Jul 01 17:00:16.091718 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.091651 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:00:16.096744 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.096717 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.115837 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.115779 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jul 01 17:00:16.167548 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.167512 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:00:16.178988 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.178946 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179001 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179042 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179091 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179112 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179134 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.179439 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.179214 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g5d98\" (UniqueName: \"kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.280630 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.280394 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.280905 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.280878 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.281017 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.280938 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.281017 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.280975 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.281192 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.281165 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-g5d98\" (UniqueName: \"kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.281277 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.281256 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.281335 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.281325 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.282175 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.282125 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.283110 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.282764 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.283212 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.283173 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.286040 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.286010 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.286887 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.286867 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.288255 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.288225 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.321186 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.321086 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g5d98\" (UniqueName: \"kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98\") pod \"console-5cbdc4bf9c-4lgtx\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:16.409131 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:16.409032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:17.813638 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:17.813599 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:17.923228 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:17.923192 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:18.018725 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:18.018689 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:19.242578 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:19.242540 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 17:00:19.243173 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:19.242604 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 17:00:20.481892 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.481866 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:00:20.484925 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:00:20.484880 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6a9efdb8_57ab_4f9a_b7fe_512475d409ad.slice/crio-1b945c9ea7a65e6a44b72ca98829d65fbe8cb592756f2d5ae4dc13ef9f923518 WatchSource:0}: Error finding container 1b945c9ea7a65e6a44b72ca98829d65fbe8cb592756f2d5ae4dc13ef9f923518: Status 404 returned error can't find the container with id 1b945c9ea7a65e6a44b72ca98829d65fbe8cb592756f2d5ae4dc13ef9f923518 Jul 01 17:00:20.819651 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.819574 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dc8866db-jtpsl" event={"ID":"23143c9e-a088-4bd7-bf69-e120cf604757","Type":"ContainerStarted","Data":"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c"} Jul 01 17:00:20.821477 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.821431 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbdc4bf9c-4lgtx" event={"ID":"6a9efdb8-57ab-4f9a-b7fe-512475d409ad","Type":"ContainerStarted","Data":"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290"} Jul 01 17:00:20.821477 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.821479 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbdc4bf9c-4lgtx" event={"ID":"6a9efdb8-57ab-4f9a-b7fe-512475d409ad","Type":"ContainerStarted","Data":"1b945c9ea7a65e6a44b72ca98829d65fbe8cb592756f2d5ae4dc13ef9f923518"} Jul 01 17:00:20.823049 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.823019 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-5499b6fb8f-9sghz" event={"ID":"0f1d0bc2-7b05-4877-82e5-e176ebe68fc9","Type":"ContainerStarted","Data":"d0553d9f0569bfde217f14dbb3859409d83688f102ee268e1da417c79e4f6f1b"} Jul 01 17:00:20.823232 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.823213 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:20.831616 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.831583 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-5499b6fb8f-9sghz" Jul 01 17:00:20.856916 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.856850 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-77dc8866db-jtpsl" podStartSLOduration=1.477808142 podStartE2EDuration="12.856827001s" totalImagesPullingTime="11.379018859s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:08 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:20.85597851 +0000 UTC m=+144.095535188" watchObservedRunningTime="2026-07-01 17:00:20.856827001 +0000 UTC m=+144.096383673" Jul 01 17:00:20.920323 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:20.920261 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/downloads-5499b6fb8f-9sghz" podStartSLOduration=2.203606845 podStartE2EDuration="20.920244736s" totalImagesPullingTime="18.716637891s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:00 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:20.919386547 +0000 UTC m=+144.158943225" watchObservedRunningTime="2026-07-01 17:00:20.920244736 +0000 UTC m=+144.159801415" Jul 01 17:00:21.021349 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:21.021287 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-5cbdc4bf9c-4lgtx" podStartSLOduration=5.021269855 podStartE2EDuration="5.021269855s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:00:16 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:00:21.020022107 +0000 UTC m=+144.259578785" watchObservedRunningTime="2026-07-01 17:00:21.021269855 +0000 UTC m=+144.260826531" Jul 01 17:00:21.145828 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:21.145719 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:21.334334 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:21.334299 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:22.236062 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:22.236016 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:22.437460 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:22.437424 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:22.536441 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:22.536360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:24.066897 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:24.066857 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:24.197255 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:24.197211 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:25.999253 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:25.999202 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:26.410227 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:26.410141 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:26.410227 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:26.410223 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:26.415862 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:26.415833 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:26.853762 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:26.853733 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:00:26.923701 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:26.923654 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:27.436538 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.436499 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-image-registry/image-registry-5f57596b68-99scq" podUID="29aca5e2-a134-4a66-9661-e00767645c69" containerName="registry" containerID="cri-o://e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337" gracePeriod=30 Jul 01 17:00:27.555688 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:00:27.555644 2580 cadvisor_stats_provider.go:567] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod29aca5e2_a134_4a66_9661_e00767645c69.slice/crio-conmon-e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337.scope\": RecentStats: unable to find data in memory cache]" Jul 01 17:00:27.749970 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.749925 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:27.755703 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.755677 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 17:00:27.777249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777212 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777268 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777292 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777342 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777370 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777404 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777442 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xnmh6\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777493 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") pod \"29aca5e2-a134-4a66-9661-e00767645c69\" (UID: \"29aca5e2-a134-4a66-9661-e00767645c69\") " Jul 01 17:00:27.777841 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777685 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:27.777905 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.777888 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:27.780664 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.780586 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:27.781854 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.781272 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6" (OuterVolumeSpecName: "kube-api-access-xnmh6") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "kube-api-access-xnmh6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:27.781854 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.781331 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:27.782049 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.781858 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:27.785094 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.785060 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:27.789569 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.789537 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "29aca5e2-a134-4a66-9661-e00767645c69" (UID: "29aca5e2-a134-4a66-9661-e00767645c69"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:00:27.839303 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.839254 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:27.855877 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.855838 2580 generic.go:356] "Generic (PLEG): container finished" podID="29aca5e2-a134-4a66-9661-e00767645c69" containerID="e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337" exitCode=0 Jul 01 17:00:27.855877 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.855876 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5f57596b68-99scq" Jul 01 17:00:27.856107 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.855903 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f57596b68-99scq" event={"ID":"29aca5e2-a134-4a66-9661-e00767645c69","Type":"ContainerDied","Data":"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337"} Jul 01 17:00:27.856107 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.855989 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5f57596b68-99scq" event={"ID":"29aca5e2-a134-4a66-9661-e00767645c69","Type":"ContainerDied","Data":"79d8ebb9e7829cbf4b0468289c2d974bfddc8018a818bd805432e1b5e9dea269"} Jul 01 17:00:27.856107 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.856012 2580 scope.go:117] "RemoveContainer" containerID="e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.884276 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885162 2580 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-bound-sa-token\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885188 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xnmh6\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-kube-api-access-xnmh6\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885203 2580 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/29aca5e2-a134-4a66-9661-e00767645c69-registry-tls\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885216 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-trusted-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885229 2580 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/29aca5e2-a134-4a66-9661-e00767645c69-ca-trust-extracted\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885241 2580 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-installation-pull-secrets\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885256 2580 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/29aca5e2-a134-4a66-9661-e00767645c69-image-registry-private-configuration\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.889825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.885270 2580 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/29aca5e2-a134-4a66-9661-e00767645c69-registry-certificates\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:27.893110 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.892972 2580 scope.go:117] "RemoveContainer" containerID="e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337" Jul 01 17:00:27.894597 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:00:27.894487 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337\": container with ID starting with e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337 not found: ID does not exist" containerID="e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337" Jul 01 17:00:27.894597 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.894543 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337"} err="failed to get container status \"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337\": rpc error: code = NotFound desc = could not find container \"e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337\": container with ID starting with e57a5274eefbfe31d3cc86ed3fd573dccbece13c5fa365c9e5e82c256bc3c337 not found: ID does not exist" Jul 01 17:00:27.898530 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:27.898497 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-5f57596b68-99scq"] Jul 01 17:00:28.649155 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:28.649120 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65448c9fc7-dtrb2_4676fb6a-72b0-4cb2-b053-133b2c4ded9c/metrics-server/0.log" Jul 01 17:00:28.743026 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:28.742986 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:29.228687 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:29.228646 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="29aca5e2-a134-4a66-9661-e00767645c69" path="/var/lib/kubelet/pods/29aca5e2-a134-4a66-9661-e00767645c69/volumes" Jul 01 17:00:30.248024 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:30.247972 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/init-textfile/0.log" Jul 01 17:00:30.448658 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:30.448635 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/node-exporter/0.log" Jul 01 17:00:30.647588 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:30.647499 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/kube-rbac-proxy/0.log" Jul 01 17:00:32.849642 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:32.849600 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-gxdxh_f4869407-5981-463b-9f9e-5ca9599b3337/prometheus-operator/0.log" Jul 01 17:00:33.046767 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:33.046734 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-gxdxh_f4869407-5981-463b-9f9e-5ca9599b3337/kube-rbac-proxy/0.log" Jul 01 17:00:33.446598 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:33.446565 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/thanos-query/0.log" Jul 01 17:00:33.657769 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:33.657741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-web/0.log" Jul 01 17:00:33.849347 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:33.849262 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy/0.log" Jul 01 17:00:34.050164 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:34.050133 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/prom-label-proxy/0.log" Jul 01 17:00:34.247293 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:34.247267 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-rules/0.log" Jul 01 17:00:34.465534 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:34.465498 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-metrics/0.log" Jul 01 17:00:34.847191 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:34.847166 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:00:35.048370 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:35.048339 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/3.log" Jul 01 17:00:35.247220 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:35.247178 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5cbdc4bf9c-4lgtx_6a9efdb8-57ab-4f9a-b7fe-512475d409ad/console/0.log" Jul 01 17:00:35.446959 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:35.446929 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-77dc8866db-jtpsl_23143c9e-a088-4bd7-bf69-e120cf604757/console/0.log" Jul 01 17:00:35.648683 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:35.648608 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-9sghz_0f1d0bc2-7b05-4877-82e5-e176ebe68fc9/download-server/0.log" Jul 01 17:00:36.706172 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:36.706137 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:36.951432 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:36.951386 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:00:39.256557 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:39.256523 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 17:00:39.260876 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:39.260846 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-65448c9fc7-dtrb2" Jul 01 17:00:51.950281 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:51.950244 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-77dc8866db-jtpsl" podUID="23143c9e-a088-4bd7-bf69-e120cf604757" containerName="console" containerID="cri-o://8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c" gracePeriod=15 Jul 01 17:00:52.236829 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.236772 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:52.289778 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.289679 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gkng6\" (UniqueName: \"kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.289998 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.289866 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.289998 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.289901 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.289998 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.289942 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.289998 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.289976 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.290202 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.290012 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config\") pod \"23143c9e-a088-4bd7-bf69-e120cf604757\" (UID: \"23143c9e-a088-4bd7-bf69-e120cf604757\") " Jul 01 17:00:52.290378 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.290340 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca" (OuterVolumeSpecName: "service-ca") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:52.290378 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.290361 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config" (OuterVolumeSpecName: "console-config") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:52.290518 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.290365 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:00:52.292153 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.292108 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:52.292267 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.292203 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:00:52.292267 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.292239 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6" (OuterVolumeSpecName: "kube-api-access-gkng6") pod "23143c9e-a088-4bd7-bf69-e120cf604757" (UID: "23143c9e-a088-4bd7-bf69-e120cf604757"). InnerVolumeSpecName "kube-api-access-gkng6". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:00:52.390745 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390710 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gkng6\" (UniqueName: \"kubernetes.io/projected/23143c9e-a088-4bd7-bf69-e120cf604757-kube-api-access-gkng6\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.390745 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390738 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.390745 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390747 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-console-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.391036 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390758 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-service-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.391036 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390768 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/23143c9e-a088-4bd7-bf69-e120cf604757-oauth-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.391036 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.390776 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/23143c9e-a088-4bd7-bf69-e120cf604757-console-oauth-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:00:52.953074 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.953042 2580 generic.go:356] "Generic (PLEG): container finished" podID="23143c9e-a088-4bd7-bf69-e120cf604757" containerID="8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c" exitCode=0 Jul 01 17:00:52.953074 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.953080 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dc8866db-jtpsl" event={"ID":"23143c9e-a088-4bd7-bf69-e120cf604757","Type":"ContainerDied","Data":"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c"} Jul 01 17:00:52.953521 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.953084 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-77dc8866db-jtpsl" Jul 01 17:00:52.953521 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.953119 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-77dc8866db-jtpsl" event={"ID":"23143c9e-a088-4bd7-bf69-e120cf604757","Type":"ContainerDied","Data":"939ed38cb7478d2c1b2b48efc837a4ef03b42957298a74f7cafcf1afd0789a29"} Jul 01 17:00:52.953521 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.953139 2580 scope.go:117] "RemoveContainer" containerID="8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c" Jul 01 17:00:52.963652 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.963634 2580 scope.go:117] "RemoveContainer" containerID="8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c" Jul 01 17:00:52.963952 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:00:52.963933 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c\": container with ID starting with 8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c not found: ID does not exist" containerID="8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c" Jul 01 17:00:52.964002 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.963962 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c"} err="failed to get container status \"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c\": rpc error: code = NotFound desc = could not find container \"8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c\": container with ID starting with 8f4052f21301138f3a51b933d7edaf6f5321b3ce46d79278dfc57910957c944c not found: ID does not exist" Jul 01 17:00:52.977486 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.977455 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:52.982627 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:52.982603 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-77dc8866db-jtpsl"] Jul 01 17:00:53.228655 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:00:53.228552 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="23143c9e-a088-4bd7-bf69-e120cf604757" path="/var/lib/kubelet/pods/23143c9e-a088-4bd7-bf69-e120cf604757/volumes" Jul 01 17:01:14.239460 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:14.239414 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:15.260083 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:15.260045 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:19.345021 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.344984 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7"] Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345253 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="29aca5e2-a134-4a66-9661-e00767645c69" containerName="registry" Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345264 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="29aca5e2-a134-4a66-9661-e00767645c69" containerName="registry" Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345274 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="23143c9e-a088-4bd7-bf69-e120cf604757" containerName="console" Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345280 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="23143c9e-a088-4bd7-bf69-e120cf604757" containerName="console" Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345335 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="29aca5e2-a134-4a66-9661-e00767645c69" containerName="registry" Jul 01 17:01:19.345512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.345344 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="23143c9e-a088-4bd7-bf69-e120cf604757" containerName="console" Jul 01 17:01:19.347597 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.347575 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.350372 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350348 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jul 01 17:01:19.350531 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350514 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jul 01 17:01:19.350608 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350514 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jul 01 17:01:19.350608 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350579 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-h6v85\"" Jul 01 17:01:19.350721 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350579 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jul 01 17:01:19.350721 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.350634 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jul 01 17:01:19.355731 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.355711 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jul 01 17:01:19.365930 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.365903 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7"] Jul 01 17:01:19.506662 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506621 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-serving-certs-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b5h5z\" (UniqueName: \"kubernetes.io/projected/375451d5-4506-4a94-af88-7e967014b1a4-kube-api-access-b5h5z\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506701 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506747 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506815 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506985 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506862 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-federate-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506985 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506887 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.506985 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.506918 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-metrics-client-ca\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.607399 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.607305 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608082 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608051 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-federate-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608211 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608105 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608211 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608151 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-metrics-client-ca\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608211 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608180 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-serving-certs-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608363 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608231 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-b5h5z\" (UniqueName: \"kubernetes.io/projected/375451d5-4506-4a94-af88-7e967014b1a4-kube-api-access-b5h5z\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608363 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608268 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.608363 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.608297 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.609347 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.609290 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-metrics-client-ca\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.609459 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.609417 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-trusted-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.609916 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.609892 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/375451d5-4506-4a94-af88-7e967014b1a4-serving-certs-ca-bundle\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.610518 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.610486 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.610738 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.610716 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-federate-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.611274 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.611248 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-telemeter-client-tls\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.611353 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.611273 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/375451d5-4506-4a94-af88-7e967014b1a4-secret-telemeter-client\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.617525 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.617501 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b5h5z\" (UniqueName: \"kubernetes.io/projected/375451d5-4506-4a94-af88-7e967014b1a4-kube-api-access-b5h5z\") pod \"telemeter-client-6f5ccc6dd5-4kmf7\" (UID: \"375451d5-4506-4a94-af88-7e967014b1a4\") " pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.658386 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.658347 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" Jul 01 17:01:19.816863 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:19.816835 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7"] Jul 01 17:01:19.818991 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:01:19.818958 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod375451d5_4506_4a94_af88_7e967014b1a4.slice/crio-6f6ed2cb70a3d9ef9895a5531ef6d1db1249485a289a95135a9d8b7fcffa5d11 WatchSource:0}: Error finding container 6f6ed2cb70a3d9ef9895a5531ef6d1db1249485a289a95135a9d8b7fcffa5d11: Status 404 returned error can't find the container with id 6f6ed2cb70a3d9ef9895a5531ef6d1db1249485a289a95135a9d8b7fcffa5d11 Jul 01 17:01:20.053601 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:20.053564 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" event={"ID":"375451d5-4506-4a94-af88-7e967014b1a4","Type":"ContainerStarted","Data":"6f6ed2cb70a3d9ef9895a5531ef6d1db1249485a289a95135a9d8b7fcffa5d11"} Jul 01 17:01:23.066190 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:23.066158 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" event={"ID":"375451d5-4506-4a94-af88-7e967014b1a4","Type":"ContainerStarted","Data":"c5f36e2bdeec31939716e77486b151e1eb491478418292aedaf0675b6536c892"} Jul 01 17:01:25.075359 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.075316 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" event={"ID":"375451d5-4506-4a94-af88-7e967014b1a4","Type":"ContainerStarted","Data":"318e9544b1f064757358813c0803f08f86caaf2b882fb39cf7abf189f7a258b7"} Jul 01 17:01:25.075776 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.075381 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" event={"ID":"375451d5-4506-4a94-af88-7e967014b1a4","Type":"ContainerStarted","Data":"4e7057cd69d016072633248f56d3677f688d5a6835f9767dccd17d9b8b8da841"} Jul 01 17:01:25.098734 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.098683 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-6f5ccc6dd5-4kmf7" podStartSLOduration=1.901804658 podStartE2EDuration="6.098667123s" totalImagesPullingTime="4.196862465s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:01:19 +0000 UTC" imagePullSessionsCount=2 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:01:25.098154907 +0000 UTC m=+208.337711596" watchObservedRunningTime="2026-07-01 17:01:25.098667123 +0000 UTC m=+208.338223800" Jul 01 17:01:25.837445 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.837410 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:01:25.840046 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.840021 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855665 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855635 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855786 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855678 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855786 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855730 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855786 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855782 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855922 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855822 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855922 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855853 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4l5gm\" (UniqueName: \"kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.855922 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.855874 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.873284 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.873247 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:01:25.956500 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956418 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-4l5gm\" (UniqueName: \"kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956690 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956528 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956690 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956570 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956690 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956606 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956882 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956708 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956882 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956766 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.956882 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.956827 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.957468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.957442 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.957576 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.957480 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.957576 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.957483 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.957722 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.957698 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.959249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.959230 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.959362 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.959344 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:25.973679 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.973648 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:25.977268 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:25.977249 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4l5gm\" (UniqueName: \"kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm\") pod \"console-68846bbb46-fhm5d\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:26.039201 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:26.039166 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:26.152043 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:26.151955 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:26.289011 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:26.288988 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:01:26.291523 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:01:26.291491 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod89f9a5f8_3fc3_40c7_b391_8287a8d56718.slice/crio-3d2ee35e74750c0d847373c4a627325dd150c226daf553e1eac8d75daf2bd7c8 WatchSource:0}: Error finding container 3d2ee35e74750c0d847373c4a627325dd150c226daf553e1eac8d75daf2bd7c8: Status 404 returned error can't find the container with id 3d2ee35e74750c0d847373c4a627325dd150c226daf553e1eac8d75daf2bd7c8 Jul 01 17:01:27.084245 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:27.084204 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68846bbb46-fhm5d" event={"ID":"89f9a5f8-3fc3-40c7-b391-8287a8d56718","Type":"ContainerStarted","Data":"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5"} Jul 01 17:01:27.084399 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:27.084264 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68846bbb46-fhm5d" event={"ID":"89f9a5f8-3fc3-40c7-b391-8287a8d56718","Type":"ContainerStarted","Data":"3d2ee35e74750c0d847373c4a627325dd150c226daf553e1eac8d75daf2bd7c8"} Jul 01 17:01:27.106231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:27.106172 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-68846bbb46-fhm5d" podStartSLOduration=2.106156035 podStartE2EDuration="2.106156035s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:01:25 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:01:27.104434049 +0000 UTC m=+210.343990726" watchObservedRunningTime="2026-07-01 17:01:27.106156035 +0000 UTC m=+210.345712712" Jul 01 17:01:36.152738 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:36.152694 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:36.153152 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:36.152767 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:36.157394 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:36.157371 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:37.123733 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:37.123644 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:01:37.180921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:37.180886 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:01:37.301848 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:37.301797 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:37.403500 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:37.403406 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:37.472427 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:37.472396 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:01:56.822446 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:01:56.822412 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:02.202196 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.202155 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-5cbdc4bf9c-4lgtx" podUID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" containerName="console" containerID="cri-o://01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290" gracePeriod=15 Jul 01 17:02:02.451168 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.451142 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:02:02.533777 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533676 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.533777 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533718 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-g5d98\" (UniqueName: \"kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.533777 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533758 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.534083 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533825 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.534083 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533882 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.534083 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533929 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.534083 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.533952 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca\") pod \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\" (UID: \"6a9efdb8-57ab-4f9a-b7fe-512475d409ad\") " Jul 01 17:02:02.534291 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.534230 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:02:02.534342 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.534285 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:02:02.534440 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.534305 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config" (OuterVolumeSpecName: "console-config") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:02:02.534493 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.534472 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca" (OuterVolumeSpecName: "service-ca") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:02:02.536035 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.536015 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:02:02.536114 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.536042 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:02:02.536114 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.536073 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98" (OuterVolumeSpecName: "kube-api-access-g5d98") pod "6a9efdb8-57ab-4f9a-b7fe-512475d409ad" (UID: "6a9efdb8-57ab-4f9a-b7fe-512475d409ad"). InnerVolumeSpecName "kube-api-access-g5d98". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:02:02.635461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635411 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-oauth-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635451 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-g5d98\" (UniqueName: \"kubernetes.io/projected/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-kube-api-access-g5d98\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635461 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-trusted-ca-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635470 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-oauth-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635479 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635756 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635487 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-console-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:02.635756 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:02.635496 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6a9efdb8-57ab-4f9a-b7fe-512475d409ad-service-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:02:03.220851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.220793 2580 generic.go:356] "Generic (PLEG): container finished" podID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" containerID="01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290" exitCode=0 Jul 01 17:02:03.220851 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.220854 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5cbdc4bf9c-4lgtx" Jul 01 17:02:03.221374 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.220877 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbdc4bf9c-4lgtx" event={"ID":"6a9efdb8-57ab-4f9a-b7fe-512475d409ad","Type":"ContainerDied","Data":"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290"} Jul 01 17:02:03.221374 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.220931 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5cbdc4bf9c-4lgtx" event={"ID":"6a9efdb8-57ab-4f9a-b7fe-512475d409ad","Type":"ContainerDied","Data":"1b945c9ea7a65e6a44b72ca98829d65fbe8cb592756f2d5ae4dc13ef9f923518"} Jul 01 17:02:03.221374 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.220951 2580 scope.go:117] "RemoveContainer" containerID="01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290" Jul 01 17:02:03.229831 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.229797 2580 scope.go:117] "RemoveContainer" containerID="01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290" Jul 01 17:02:03.230133 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:02:03.230107 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290\": container with ID starting with 01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290 not found: ID does not exist" containerID="01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290" Jul 01 17:02:03.230201 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.230148 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290"} err="failed to get container status \"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290\": rpc error: code = NotFound desc = could not find container \"01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290\": container with ID starting with 01e72007fe970c11972c6a4d2456532787bb476474df646c66d3b3159abed290 not found: ID does not exist" Jul 01 17:02:03.245200 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.245170 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:02:03.247500 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:03.247476 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5cbdc4bf9c-4lgtx"] Jul 01 17:02:05.229569 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:05.229528 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" path="/var/lib/kubelet/pods/6a9efdb8-57ab-4f9a-b7fe-512475d409ad/volumes" Jul 01 17:02:29.612857 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:29.612772 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:29.712642 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:29.712611 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:34.597029 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.596986 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:02:34.597466 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.597450 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" containerName="console" Jul 01 17:02:34.597466 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.597468 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" containerName="console" Jul 01 17:02:34.597546 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.597515 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="6a9efdb8-57ab-4f9a-b7fe-512475d409ad" containerName="console" Jul 01 17:02:34.599493 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.599473 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.615913 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.615863 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:02:34.669009 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.668958 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:34.698201 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698162 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698375 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698209 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698375 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698267 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698375 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698313 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698375 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698348 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698511 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698430 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.698511 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.698451 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-798kh\" (UniqueName: \"kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.765928 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.765894 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:34.799154 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799111 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799199 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799238 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799278 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799310 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.799348 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.799337 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-798kh\" (UniqueName: \"kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.800100 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.800068 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.800239 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.800113 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.800239 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.800129 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.800239 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.800171 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.801757 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.801728 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.801914 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.801878 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.809058 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.809032 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-798kh\" (UniqueName: \"kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh\") pod \"console-57c9d4678b-zjvcn\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:34.910722 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:34.910629 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:35.055512 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.055481 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:02:35.058597 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:02:35.058566 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod33eba021_7869_4138_babe_81d642f6fcdd.slice/crio-6fa605d4310a5d5802fc9503e5f533ebb64fd183bc3e31a82244de67ed20286a WatchSource:0}: Error finding container 6fa605d4310a5d5802fc9503e5f533ebb64fd183bc3e31a82244de67ed20286a: Status 404 returned error can't find the container with id 6fa605d4310a5d5802fc9503e5f533ebb64fd183bc3e31a82244de67ed20286a Jul 01 17:02:35.166096 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.166008 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:35.339408 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.339369 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9d4678b-zjvcn" event={"ID":"33eba021-7869-4138-babe-81d642f6fcdd","Type":"ContainerStarted","Data":"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7"} Jul 01 17:02:35.339612 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.339420 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9d4678b-zjvcn" event={"ID":"33eba021-7869-4138-babe-81d642f6fcdd","Type":"ContainerStarted","Data":"6fa605d4310a5d5802fc9503e5f533ebb64fd183bc3e31a82244de67ed20286a"} Jul 01 17:02:35.363828 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.360350 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-57c9d4678b-zjvcn" podStartSLOduration=1.360329294 podStartE2EDuration="1.360329294s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:02:34 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:02:35.357577555 +0000 UTC m=+278.597134265" watchObservedRunningTime="2026-07-01 17:02:35.360329294 +0000 UTC m=+278.599885975" Jul 01 17:02:35.398727 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.398689 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:35.536433 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:35.536395 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:41.604917 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:41.604874 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:41.709238 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:41.709207 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:44.910774 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:44.910734 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:44.911193 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:44.910822 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:44.915606 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:44.915582 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:45.383782 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:45.383753 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:02:45.436003 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:45.435970 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:02:45.544995 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:45.544951 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:45.698919 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:45.698877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:45.782262 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:45.782228 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:54.029307 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:54.029265 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:54.191936 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:54.191901 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:54.427998 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:54.427908 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:02:57.149523 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:57.149489 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:02:57.150089 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:57.150072 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:02:57.160106 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:02:57.160079 2580 kubelet.go:1648] "Image garbage collection succeeded" Jul 01 17:03:10.463022 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.462965 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-68846bbb46-fhm5d" podUID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" containerName="console" containerID="cri-o://26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5" gracePeriod=15 Jul 01 17:03:10.700836 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.700787 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:03:10.805910 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.805788 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.805910 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.805864 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806165 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.805916 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806165 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.805963 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806165 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.805990 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806165 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806018 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806165 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806118 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4l5gm\" (UniqueName: \"kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm\") pod \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\" (UID: \"89f9a5f8-3fc3-40c7-b391-8287a8d56718\") " Jul 01 17:03:10.806473 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806448 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca" (OuterVolumeSpecName: "service-ca") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:03:10.806535 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806493 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:03:10.806535 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806517 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config" (OuterVolumeSpecName: "console-config") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:03:10.806608 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.806588 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:03:10.807995 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.807968 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:03:10.808104 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.808086 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:03:10.808418 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.808395 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm" (OuterVolumeSpecName: "kube-api-access-4l5gm") pod "89f9a5f8-3fc3-40c7-b391-8287a8d56718" (UID: "89f9a5f8-3fc3-40c7-b391-8287a8d56718"). InnerVolumeSpecName "kube-api-access-4l5gm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:03:10.907231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907184 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907219 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-oauth-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907229 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-oauth-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907238 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4l5gm\" (UniqueName: \"kubernetes.io/projected/89f9a5f8-3fc3-40c7-b391-8287a8d56718-kube-api-access-4l5gm\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907247 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/89f9a5f8-3fc3-40c7-b391-8287a8d56718-console-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907523 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907255 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-service-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:10.907523 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:10.907262 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/89f9a5f8-3fc3-40c7-b391-8287a8d56718-trusted-ca-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:03:11.477042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.477007 2580 generic.go:356] "Generic (PLEG): container finished" podID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" containerID="26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5" exitCode=0 Jul 01 17:03:11.477454 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.477068 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-68846bbb46-fhm5d" Jul 01 17:03:11.477454 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.477076 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68846bbb46-fhm5d" event={"ID":"89f9a5f8-3fc3-40c7-b391-8287a8d56718","Type":"ContainerDied","Data":"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5"} Jul 01 17:03:11.477454 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.477114 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-68846bbb46-fhm5d" event={"ID":"89f9a5f8-3fc3-40c7-b391-8287a8d56718","Type":"ContainerDied","Data":"3d2ee35e74750c0d847373c4a627325dd150c226daf553e1eac8d75daf2bd7c8"} Jul 01 17:03:11.477454 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.477134 2580 scope.go:117] "RemoveContainer" containerID="26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5" Jul 01 17:03:11.485553 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.485530 2580 scope.go:117] "RemoveContainer" containerID="26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5" Jul 01 17:03:11.485866 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:03:11.485844 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5\": container with ID starting with 26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5 not found: ID does not exist" containerID="26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5" Jul 01 17:03:11.485948 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.485874 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5"} err="failed to get container status \"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5\": rpc error: code = NotFound desc = could not find container \"26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5\": container with ID starting with 26a271e10e960fbbeae9dc8df8704f67ac114b5c013818f00d504a842ec588f5 not found: ID does not exist" Jul 01 17:03:11.497825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.497720 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:03:11.504681 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:11.504654 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-68846bbb46-fhm5d"] Jul 01 17:03:13.228439 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:13.228406 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" path="/var/lib/kubelet/pods/89f9a5f8-3fc3-40c7-b391-8287a8d56718/volumes" Jul 01 17:03:44.652645 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:44.652609 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:03:44.742004 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:44.741952 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:03:44.960519 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:44.960476 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:03:45.064063 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:45.064021 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:03:45.163169 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:45.163116 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:03:46.113700 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.113661 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs"] Jul 01 17:03:46.114101 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.113975 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" containerName="console" Jul 01 17:03:46.114101 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.113986 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" containerName="console" Jul 01 17:03:46.114101 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.114037 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="89f9a5f8-3fc3-40c7-b391-8287a8d56718" containerName="console" Jul 01 17:03:46.117051 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.117033 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.119622 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.119602 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jul 01 17:03:46.119738 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.119664 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jul 01 17:03:46.120677 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.120657 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-8lglx\"" Jul 01 17:03:46.130826 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.130783 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs"] Jul 01 17:03:46.159166 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.159129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.159166 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.159169 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.159396 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.159241 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-58kdf\" (UniqueName: \"kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.260213 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.260169 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.260213 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.260209 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.260473 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.260257 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-58kdf\" (UniqueName: \"kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.260675 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.260652 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.260724 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.260675 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.269492 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.269456 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-58kdf\" (UniqueName: \"kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf\") pod \"74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.426182 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.426090 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:03:46.558368 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.558339 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs"] Jul 01 17:03:46.560240 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:03:46.560212 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf5206fbf_34df_4da4_aa08_4f4ff6bf696a.slice/crio-b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6 WatchSource:0}: Error finding container b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6: Status 404 returned error can't find the container with id b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6 Jul 01 17:03:46.562347 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.562326 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 17:03:46.611932 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:46.611889 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" event={"ID":"f5206fbf-34df-4da4-aa08-4f4ff6bf696a","Type":"ContainerStarted","Data":"b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6"} Jul 01 17:03:54.644368 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:54.644328 2580 generic.go:356] "Generic (PLEG): container finished" podID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerID="f90417acc4b3253afb75e886ce076f9622c0657a846ae4a634423488fe20e7cb" exitCode=0 Jul 01 17:03:54.644829 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:54.644404 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" event={"ID":"f5206fbf-34df-4da4-aa08-4f4ff6bf696a","Type":"ContainerDied","Data":"f90417acc4b3253afb75e886ce076f9622c0657a846ae4a634423488fe20e7cb"} Jul 01 17:03:56.653997 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:56.653906 2580 generic.go:356] "Generic (PLEG): container finished" podID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerID="18d087493cd11fad39fce9525f8797d9bb1e2a63921f706f9284d5141fa5148e" exitCode=0 Jul 01 17:03:56.654365 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:03:56.654004 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" event={"ID":"f5206fbf-34df-4da4-aa08-4f4ff6bf696a","Type":"ContainerDied","Data":"18d087493cd11fad39fce9525f8797d9bb1e2a63921f706f9284d5141fa5148e"} Jul 01 17:04:07.702306 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:07.702269 2580 generic.go:356] "Generic (PLEG): container finished" podID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerID="bafee6f5378019f7418a0a7170ad8e337caf3d85d87ff60399c1c24944e8930b" exitCode=0 Jul 01 17:04:07.702673 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:07.702317 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" event={"ID":"f5206fbf-34df-4da4-aa08-4f4ff6bf696a","Type":"ContainerDied","Data":"bafee6f5378019f7418a0a7170ad8e337caf3d85d87ff60399c1c24944e8930b"} Jul 01 17:04:08.871473 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.871434 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:04:08.955935 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.955895 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle\") pod \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " Jul 01 17:04:08.956136 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.955953 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util\") pod \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " Jul 01 17:04:08.956136 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.956000 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-58kdf\" (UniqueName: \"kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf\") pod \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\" (UID: \"f5206fbf-34df-4da4-aa08-4f4ff6bf696a\") " Jul 01 17:04:08.956527 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.956500 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle" (OuterVolumeSpecName: "bundle") pod "f5206fbf-34df-4da4-aa08-4f4ff6bf696a" (UID: "f5206fbf-34df-4da4-aa08-4f4ff6bf696a"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:04:08.958306 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.958277 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf" (OuterVolumeSpecName: "kube-api-access-58kdf") pod "f5206fbf-34df-4da4-aa08-4f4ff6bf696a" (UID: "f5206fbf-34df-4da4-aa08-4f4ff6bf696a"). InnerVolumeSpecName "kube-api-access-58kdf". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:04:08.962576 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:08.962546 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util" (OuterVolumeSpecName: "util") pod "f5206fbf-34df-4da4-aa08-4f4ff6bf696a" (UID: "f5206fbf-34df-4da4-aa08-4f4ff6bf696a"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jul 01 17:04:09.056561 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.056523 2580 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:04:09.056561 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.056555 2580 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-util\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:04:09.056561 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.056564 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-58kdf\" (UniqueName: \"kubernetes.io/projected/f5206fbf-34df-4da4-aa08-4f4ff6bf696a-kube-api-access-58kdf\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:04:09.710620 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.710587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" event={"ID":"f5206fbf-34df-4da4-aa08-4f4ff6bf696a","Type":"ContainerDied","Data":"b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6"} Jul 01 17:04:09.710793 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.710632 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/74e94af6ec69867d65f1d90648440bdc4f0845d5e32c4818a460ebb589mzrhs" Jul 01 17:04:09.710793 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:09.710634 2580 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="b96814aae7a4b3fe26329cd531315f5e82e5642da8f33267f192dad0f4fbbea6" Jul 01 17:04:13.193145 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:13.193104 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:15.148794 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.148752 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74"] Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149185 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="pull" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149202 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="pull" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149213 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="extract" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149220 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="extract" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149243 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="util" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149251 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="util" Jul 01 17:04:15.149316 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.149314 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="f5206fbf-34df-4da4-aa08-4f4ff6bf696a" containerName="extract" Jul 01 17:04:15.151589 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.151569 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.158144 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.158125 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jul 01 17:04:15.158248 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.158172 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jul 01 17:04:15.158248 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.158190 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jul 01 17:04:15.158357 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.158273 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-77jrj\"" Jul 01 17:04:15.177088 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.177059 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74"] Jul 01 17:04:15.312324 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.312278 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j9x9g\" (UniqueName: \"kubernetes.io/projected/725aaed5-1133-47aa-a99c-db8a718d5c14-kube-api-access-j9x9g\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.312324 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.312323 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/725aaed5-1133-47aa-a99c-db8a718d5c14-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.413795 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.413699 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/725aaed5-1133-47aa-a99c-db8a718d5c14-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.413997 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.413880 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-j9x9g\" (UniqueName: \"kubernetes.io/projected/725aaed5-1133-47aa-a99c-db8a718d5c14-kube-api-access-j9x9g\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.416563 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.416521 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/725aaed5-1133-47aa-a99c-db8a718d5c14-certificates\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.428036 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.428012 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j9x9g\" (UniqueName: \"kubernetes.io/projected/725aaed5-1133-47aa-a99c-db8a718d5c14-kube-api-access-j9x9g\") pod \"custom-metrics-autoscaler-operator-75f7c5f8f-r8c74\" (UID: \"725aaed5-1133-47aa-a99c-db8a718d5c14\") " pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.461059 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.461016 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:15.512772 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.512578 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:15.636713 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.636690 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74"] Jul 01 17:04:15.638645 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:04:15.638616 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod725aaed5_1133_47aa_a99c_db8a718d5c14.slice/crio-caf54e291802d152f7aefc9e915bdc0bcd4a7a81803c05eccdaaaf794cedec11 WatchSource:0}: Error finding container caf54e291802d152f7aefc9e915bdc0bcd4a7a81803c05eccdaaaf794cedec11: Status 404 returned error can't find the container with id caf54e291802d152f7aefc9e915bdc0bcd4a7a81803c05eccdaaaf794cedec11 Jul 01 17:04:15.734478 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:15.734446 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" event={"ID":"725aaed5-1133-47aa-a99c-db8a718d5c14","Type":"ContainerStarted","Data":"caf54e291802d152f7aefc9e915bdc0bcd4a7a81803c05eccdaaaf794cedec11"} Jul 01 17:04:18.749624 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:18.749587 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" event={"ID":"725aaed5-1133-47aa-a99c-db8a718d5c14","Type":"ContainerStarted","Data":"b00faf1a72c1b7ea3d7924fb403f3a8462c796c03a36b9fd204cc5cf6e336f3d"} Jul 01 17:04:18.750126 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:18.749718 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:18.785646 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:18.785571 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" podStartSLOduration=0.791974861 podStartE2EDuration="3.785554559s" totalImagesPullingTime="2.993579698s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:04:15 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:04:18.783009353 +0000 UTC m=+382.022566029" watchObservedRunningTime="2026-07-01 17:04:18.785554559 +0000 UTC m=+382.025111235" Jul 01 17:04:19.185821 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.185779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:19.537794 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.537757 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:19.552734 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.552707 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-68bff59c-t77lr"] Jul 01 17:04:19.554823 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.554784 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.557759 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.557736 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-f9shp\"" Jul 01 17:04:19.557893 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.557790 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jul 01 17:04:19.558040 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.558019 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jul 01 17:04:19.566285 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.566262 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-t77lr"] Jul 01 17:04:19.639705 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.639671 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:19.651009 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.650977 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2t2dz\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-kube-api-access-2t2dz\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.651009 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.651011 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.651209 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.651034 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/38d9553e-61c8-44fe-aa34-58d22c9a4630-cabundle0\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.751710 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.751669 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/38d9553e-61c8-44fe-aa34-58d22c9a4630-cabundle0\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.751776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-2t2dz\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-kube-api-access-2t2dz\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.751834 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:19.751943 2580 secret.go:281] references non-existent secret key: ca.crt Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:19.751960 2580 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:19.751971 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-t77lr: references non-existent secret key: ca.crt Jul 01 17:04:19.752208 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:19.752040 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates podName:38d9553e-61c8-44fe-aa34-58d22c9a4630 nodeName:}" failed. No retries permitted until 2026-07-01 17:04:20.25201984 +0000 UTC m=+383.491576502 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates") pod "keda-operator-68bff59c-t77lr" (UID: "38d9553e-61c8-44fe-aa34-58d22c9a4630") : references non-existent secret key: ca.crt Jul 01 17:04:19.752499 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.752452 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/38d9553e-61c8-44fe-aa34-58d22c9a4630-cabundle0\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.770109 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.770080 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2t2dz\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-kube-api-access-2t2dz\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:19.913905 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:19.913777 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:20.024264 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.024229 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j"] Jul 01 17:04:20.026480 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.026463 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.033027 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.032998 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jul 01 17:04:20.043795 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.043757 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j"] Jul 01 17:04:20.064059 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.064028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:20.154727 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.154689 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.154978 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.154744 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x4zgb\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-kube-api-access-x4zgb\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.154978 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.154854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1d0f6609-acfc-44a1-a743-98206142af8e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.255852 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.255780 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.255902 2580 secret.go:281] references non-existent secret key: tls.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.255928 2580 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.255950 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j: references non-existent secret key: tls.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.255903 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.255999 2580 secret.go:281] references non-existent secret key: ca.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.256013 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates podName:1d0f6609-acfc-44a1-a743-98206142af8e nodeName:}" failed. No retries permitted until 2026-07-01 17:04:20.755994117 +0000 UTC m=+383.995550775 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates") pod "keda-metrics-apiserver-559f75f947-kvq9j" (UID: "1d0f6609-acfc-44a1-a743-98206142af8e") : references non-existent secret key: tls.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.256017 2580 projected.go:279] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jul 01 17:04:20.256023 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.256028 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-68bff59c-t77lr: references non-existent secret key: ca.crt Jul 01 17:04:20.256445 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.256032 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-x4zgb\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-kube-api-access-x4zgb\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.256445 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.256079 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates podName:38d9553e-61c8-44fe-aa34-58d22c9a4630 nodeName:}" failed. No retries permitted until 2026-07-01 17:04:21.25606281 +0000 UTC m=+384.495619465 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates") pod "keda-operator-68bff59c-t77lr" (UID: "38d9553e-61c8-44fe-aa34-58d22c9a4630") : references non-existent secret key: ca.crt Jul 01 17:04:20.256445 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.256168 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1d0f6609-acfc-44a1-a743-98206142af8e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.256607 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.256470 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/1d0f6609-acfc-44a1-a743-98206142af8e-temp-vol\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.270921 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.270897 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x4zgb\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-kube-api-access-x4zgb\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.663385 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.663294 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-7wspd"] Jul 01 17:04:20.665544 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.665520 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.668578 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.668557 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jul 01 17:04:20.702475 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.702443 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-7wspd"] Jul 01 17:04:20.761667 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.761634 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nntlq\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-kube-api-access-nntlq\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.762135 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.761691 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.762135 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.761796 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.764601 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.764577 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/1d0f6609-acfc-44a1-a743-98206142af8e-certificates\") pod \"keda-metrics-apiserver-559f75f947-kvq9j\" (UID: \"1d0f6609-acfc-44a1-a743-98206142af8e\") " pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:20.862645 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.862602 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nntlq\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-kube-api-access-nntlq\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.862850 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.862666 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.862850 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.862775 2580 projected.go:266] Couldn't get secret openshift-keda/keda-admission-webhooks-certs: secret "keda-admission-webhooks-certs" not found Jul 01 17:04:20.862850 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.862833 2580 projected.go:196] Error preparing data for projected volume certificates for pod openshift-keda/keda-admission-5fbd5c4644-7wspd: secret "keda-admission-webhooks-certs" not found Jul 01 17:04:20.862999 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:04:20.862905 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates podName:dd8a64e8-bc1d-4095-8881-bbc9eeadede2 nodeName:}" failed. No retries permitted until 2026-07-01 17:04:21.362884854 +0000 UTC m=+384.602441528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates") pod "keda-admission-5fbd5c4644-7wspd" (UID: "dd8a64e8-bc1d-4095-8881-bbc9eeadede2") : secret "keda-admission-webhooks-certs" not found Jul 01 17:04:20.875033 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.874999 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nntlq\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-kube-api-access-nntlq\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:20.936753 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:20.936658 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:21.013360 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.013319 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.101931 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.101908 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j"] Jul 01 17:04:21.104656 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:04:21.104627 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1d0f6609_acfc_44a1_a743_98206142af8e.slice/crio-1e9442b9f8db1cc10f330bbe3c26c4bd85e7a7cdbb9a08b6b43d28b295037392 WatchSource:0}: Error finding container 1e9442b9f8db1cc10f330bbe3c26c4bd85e7a7cdbb9a08b6b43d28b295037392: Status 404 returned error can't find the container with id 1e9442b9f8db1cc10f330bbe3c26c4bd85e7a7cdbb9a08b6b43d28b295037392 Jul 01 17:04:21.105438 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.105370 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.239884 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.239843 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.266177 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.266145 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:21.268602 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.268553 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/38d9553e-61c8-44fe-aa34-58d22c9a4630-certificates\") pod \"keda-operator-68bff59c-t77lr\" (UID: \"38d9553e-61c8-44fe-aa34-58d22c9a4630\") " pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:21.342197 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.342157 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.365049 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.365018 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:21.367392 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.367365 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:21.369792 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.369771 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/dd8a64e8-bc1d-4095-8881-bbc9eeadede2-certificates\") pod \"keda-admission-5fbd5c4644-7wspd\" (UID: \"dd8a64e8-bc1d-4095-8881-bbc9eeadede2\") " pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:21.480071 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.480028 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.509665 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.509589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-68bff59c-t77lr"] Jul 01 17:04:21.512889 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:04:21.512860 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod38d9553e_61c8_44fe_aa34_58d22c9a4630.slice/crio-b15d1fbcbe67f82289f07a3178c87cecebc83adc305bf0b9bcf7f7949dd6e09c WatchSource:0}: Error finding container b15d1fbcbe67f82289f07a3178c87cecebc83adc305bf0b9bcf7f7949dd6e09c: Status 404 returned error can't find the container with id b15d1fbcbe67f82289f07a3178c87cecebc83adc305bf0b9bcf7f7949dd6e09c Jul 01 17:04:21.568476 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.568440 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:21.578511 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.578476 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:21.747970 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.747940 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-5fbd5c4644-7wspd"] Jul 01 17:04:21.750142 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:04:21.750110 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddd8a64e8_bc1d_4095_8881_bbc9eeadede2.slice/crio-16ac1d2785da1ace033d3b36737b83837dc2f0f872f62cef57d8e6f12284e4b9 WatchSource:0}: Error finding container 16ac1d2785da1ace033d3b36737b83837dc2f0f872f62cef57d8e6f12284e4b9: Status 404 returned error can't find the container with id 16ac1d2785da1ace033d3b36737b83837dc2f0f872f62cef57d8e6f12284e4b9 Jul 01 17:04:21.764380 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.764303 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" event={"ID":"dd8a64e8-bc1d-4095-8881-bbc9eeadede2","Type":"ContainerStarted","Data":"16ac1d2785da1ace033d3b36737b83837dc2f0f872f62cef57d8e6f12284e4b9"} Jul 01 17:04:21.765658 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.765627 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-t77lr" event={"ID":"38d9553e-61c8-44fe-aa34-58d22c9a4630","Type":"ContainerStarted","Data":"b15d1fbcbe67f82289f07a3178c87cecebc83adc305bf0b9bcf7f7949dd6e09c"} Jul 01 17:04:21.766743 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:21.766717 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" event={"ID":"1d0f6609-acfc-44a1-a743-98206142af8e","Type":"ContainerStarted","Data":"1e9442b9f8db1cc10f330bbe3c26c4bd85e7a7cdbb9a08b6b43d28b295037392"} Jul 01 17:04:24.782472 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.782406 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" event={"ID":"1d0f6609-acfc-44a1-a743-98206142af8e","Type":"ContainerStarted","Data":"61dc651cec49ab541ff02b65f9c86d4570e748669abf850a768052fb1ca4fbd5"} Jul 01 17:04:24.782963 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.782660 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:24.784155 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.784121 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" event={"ID":"dd8a64e8-bc1d-4095-8881-bbc9eeadede2","Type":"ContainerStarted","Data":"09b9342a5f4ff15984ea13c5af91a873505a7d28183b030e556ef27a8677a283"} Jul 01 17:04:24.784277 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.784185 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:24.805232 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.805165 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" podStartSLOduration=2.180828658 podStartE2EDuration="4.805144919s" totalImagesPullingTime="2.624316261s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:04:24.802983806 +0000 UTC m=+388.042540483" watchObservedRunningTime="2026-07-01 17:04:24.805144919 +0000 UTC m=+388.044701598" Jul 01 17:04:24.823697 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:24.823639 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" podStartSLOduration=2.846192583 podStartE2EDuration="4.823623036s" totalImagesPullingTime="1.977430453s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:04:20 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:04:24.822633342 +0000 UTC m=+388.062190021" watchObservedRunningTime="2026-07-01 17:04:24.823623036 +0000 UTC m=+388.063179709" Jul 01 17:04:26.795163 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:26.795127 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-68bff59c-t77lr" event={"ID":"38d9553e-61c8-44fe-aa34-58d22c9a4630","Type":"ContainerStarted","Data":"8d7849bc862fa4866615553e01ddd80faa36d9b87361d185c91d505d5d77a58b"} Jul 01 17:04:26.795585 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:26.795233 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:04:26.822231 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:26.822180 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-keda/keda-operator-68bff59c-t77lr" podStartSLOduration=3.287911183 podStartE2EDuration="7.82216359s" totalImagesPullingTime="4.534252407s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:04:19 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:04:26.820947152 +0000 UTC m=+390.060503829" watchObservedRunningTime="2026-07-01 17:04:26.82216359 +0000 UTC m=+390.061720267" Jul 01 17:04:35.794671 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:35.794642 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-559f75f947-kvq9j" Jul 01 17:04:39.245375 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:39.245338 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:39.375224 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:39.375186 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:04:39.756381 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:39.756353 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-75f7c5f8f-r8c74" Jul 01 17:04:45.793087 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:45.793056 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-5fbd5c4644-7wspd" Jul 01 17:04:47.801191 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:04:47.801159 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-68bff59c-t77lr" Jul 01 17:05:30.226361 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.226279 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:30.289683 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.289645 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:05:30.292345 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.292322 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.294845 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.294820 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jul 01 17:05:30.295943 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.295674 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jul 01 17:05:30.295943 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.295719 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-sjpcg\"" Jul 01 17:05:30.295943 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.295718 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jul 01 17:05:30.306612 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.306589 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:05:30.334370 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.334335 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:30.419614 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.419574 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.419784 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.419669 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tvqbw\" (UniqueName: \"kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.431671 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.431630 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:30.520293 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.520198 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.520293 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.520266 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tvqbw\" (UniqueName: \"kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.520497 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:05:30.520346 2580 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jul 01 17:05:30.520497 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:05:30.520409 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert podName:a5b93154-b5c4-42aa-b305-a719e3254d82 nodeName:}" failed. No retries permitted until 2026-07-01 17:05:31.020391804 +0000 UTC m=+454.259948459 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert") pod "kserve-controller-manager-84f4454459-rpnzs" (UID: "a5b93154-b5c4-42aa-b305-a719e3254d82") : secret "kserve-webhook-server-cert" not found Jul 01 17:05:30.533200 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.533161 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tvqbw\" (UniqueName: \"kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:30.866256 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:30.866172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:31.026249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.026208 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:31.028892 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.028864 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") pod \"kserve-controller-manager-84f4454459-rpnzs\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:31.205125 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.205093 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:31.268331 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.268294 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:31.352791 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.352759 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:05:31.354565 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:05:31.354538 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5b93154_b5c4_42aa_b305_a719e3254d82.slice/crio-e3bc8fcc3bf15c89e87b7ac3d7a0311cee5ddcb787c84640a3e24f7ee2e9e381 WatchSource:0}: Error finding container e3bc8fcc3bf15c89e87b7ac3d7a0311cee5ddcb787c84640a3e24f7ee2e9e381: Status 404 returned error can't find the container with id e3bc8fcc3bf15c89e87b7ac3d7a0311cee5ddcb787c84640a3e24f7ee2e9e381 Jul 01 17:05:31.389091 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.389060 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:31.509005 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:31.508919 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:05:32.075795 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:32.075756 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" event={"ID":"a5b93154-b5c4-42aa-b305-a719e3254d82","Type":"ContainerStarted","Data":"e3bc8fcc3bf15c89e87b7ac3d7a0311cee5ddcb787c84640a3e24f7ee2e9e381"} Jul 01 17:05:34.086204 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:34.086170 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" event={"ID":"a5b93154-b5c4-42aa-b305-a719e3254d82","Type":"ContainerStarted","Data":"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e"} Jul 01 17:05:34.086562 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:34.086290 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:05:34.128298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:34.128248 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" podStartSLOduration=1.583375352 podStartE2EDuration="4.128231615s" totalImagesPullingTime="2.544856263s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:05:30 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:05:34.127708408 +0000 UTC m=+457.367265086" watchObservedRunningTime="2026-07-01 17:05:34.128231615 +0000 UTC m=+457.367788292" Jul 01 17:05:40.826666 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:05:40.826624 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:05.095228 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:05.095198 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:06:06.413709 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.413675 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:06:06.414109 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.413918 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" podUID="a5b93154-b5c4-42aa-b305-a719e3254d82" containerName="manager" containerID="cri-o://5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e" gracePeriod=10 Jul 01 17:06:06.462469 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.462437 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-84f4454459-s5vgc"] Jul 01 17:06:06.464585 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.464569 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.480058 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.480029 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-s5vgc"] Jul 01 17:06:06.521261 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.521218 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w6c5x\" (UniqueName: \"kubernetes.io/projected/1a0b5809-49a0-40e3-864b-014af014bd39-kube-api-access-w6c5x\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.521261 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.521258 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a0b5809-49a0-40e3-864b-014af014bd39-cert\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.622505 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.622414 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-w6c5x\" (UniqueName: \"kubernetes.io/projected/1a0b5809-49a0-40e3-864b-014af014bd39-kube-api-access-w6c5x\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.622683 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.622535 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a0b5809-49a0-40e3-864b-014af014bd39-cert\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.624901 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.624872 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1a0b5809-49a0-40e3-864b-014af014bd39-cert\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.632772 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.632747 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w6c5x\" (UniqueName: \"kubernetes.io/projected/1a0b5809-49a0-40e3-864b-014af014bd39-kube-api-access-w6c5x\") pod \"kserve-controller-manager-84f4454459-s5vgc\" (UID: \"1a0b5809-49a0-40e3-864b-014af014bd39\") " pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.650682 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.650657 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:06:06.723492 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.723463 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tvqbw\" (UniqueName: \"kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw\") pod \"a5b93154-b5c4-42aa-b305-a719e3254d82\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " Jul 01 17:06:06.723680 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.723527 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") pod \"a5b93154-b5c4-42aa-b305-a719e3254d82\" (UID: \"a5b93154-b5c4-42aa-b305-a719e3254d82\") " Jul 01 17:06:06.725644 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.725608 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert" (OuterVolumeSpecName: "cert") pod "a5b93154-b5c4-42aa-b305-a719e3254d82" (UID: "a5b93154-b5c4-42aa-b305-a719e3254d82"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:06:06.725756 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.725678 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw" (OuterVolumeSpecName: "kube-api-access-tvqbw") pod "a5b93154-b5c4-42aa-b305-a719e3254d82" (UID: "a5b93154-b5c4-42aa-b305-a719e3254d82"). InnerVolumeSpecName "kube-api-access-tvqbw". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:06:06.812032 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.811990 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:06.824461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.824427 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tvqbw\" (UniqueName: \"kubernetes.io/projected/a5b93154-b5c4-42aa-b305-a719e3254d82-kube-api-access-tvqbw\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:06:06.824461 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.824453 2580 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/a5b93154-b5c4-42aa-b305-a719e3254d82-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:06:06.947966 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:06.947912 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-s5vgc"] Jul 01 17:06:06.950077 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:06:06.950049 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1a0b5809_49a0_40e3_864b_014af014bd39.slice/crio-275f1d4a1ad882b0c5baada7af98f52c148faf24e706223d90ad908df0d25917 WatchSource:0}: Error finding container 275f1d4a1ad882b0c5baada7af98f52c148faf24e706223d90ad908df0d25917: Status 404 returned error can't find the container with id 275f1d4a1ad882b0c5baada7af98f52c148faf24e706223d90ad908df0d25917 Jul 01 17:06:07.225602 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.225572 2580 generic.go:356] "Generic (PLEG): container finished" podID="a5b93154-b5c4-42aa-b305-a719e3254d82" containerID="5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e" exitCode=0 Jul 01 17:06:07.226496 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.226470 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" Jul 01 17:06:07.229867 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.229838 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" event={"ID":"a5b93154-b5c4-42aa-b305-a719e3254d82","Type":"ContainerDied","Data":"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e"} Jul 01 17:06:07.229983 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.229893 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-rpnzs" event={"ID":"a5b93154-b5c4-42aa-b305-a719e3254d82","Type":"ContainerDied","Data":"e3bc8fcc3bf15c89e87b7ac3d7a0311cee5ddcb787c84640a3e24f7ee2e9e381"} Jul 01 17:06:07.229983 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.229918 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" event={"ID":"1a0b5809-49a0-40e3-864b-014af014bd39","Type":"ContainerStarted","Data":"275f1d4a1ad882b0c5baada7af98f52c148faf24e706223d90ad908df0d25917"} Jul 01 17:06:07.229983 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.229950 2580 scope.go:117] "RemoveContainer" containerID="5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e" Jul 01 17:06:07.238878 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.238859 2580 scope.go:117] "RemoveContainer" containerID="5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e" Jul 01 17:06:07.239187 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:06:07.239164 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e\": container with ID starting with 5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e not found: ID does not exist" containerID="5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e" Jul 01 17:06:07.239273 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.239194 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e"} err="failed to get container status \"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e\": rpc error: code = NotFound desc = could not find container \"5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e\": container with ID starting with 5c980cd656a3e2212e988f9724480250fe3f689b0647b583ab9d6c4ef4cd024e not found: ID does not exist" Jul 01 17:06:07.263935 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.263902 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:06:07.265874 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.265850 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-84f4454459-rpnzs"] Jul 01 17:06:07.496364 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:07.496290 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:08.233150 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:08.233113 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" event={"ID":"1a0b5809-49a0-40e3-864b-014af014bd39","Type":"ContainerStarted","Data":"548d2f80409dce3a0f45606c2987cd7839b83fd258b29733a34493fa7acf08f7"} Jul 01 17:06:08.233328 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:08.233237 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:08.254769 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:08.254717 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" podStartSLOduration=1.814883969 podStartE2EDuration="2.254699567s" totalImagesPullingTime="439.815598ms" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:06:06 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:06:08.253535011 +0000 UTC m=+491.493091700" watchObservedRunningTime="2026-07-01 17:06:08.254699567 +0000 UTC m=+491.494256244" Jul 01 17:06:09.234991 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:09.234953 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5b93154-b5c4-42aa-b305-a719e3254d82" path="/var/lib/kubelet/pods/a5b93154-b5c4-42aa-b305-a719e3254d82/volumes" Jul 01 17:06:30.115632 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.115598 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-console/console-67d6987486-889cc"] Jul 01 17:06:30.116171 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.116103 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5b93154-b5c4-42aa-b305-a719e3254d82" containerName="manager" Jul 01 17:06:30.116171 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.116139 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5b93154-b5c4-42aa-b305-a719e3254d82" containerName="manager" Jul 01 17:06:30.116308 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.116248 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="a5b93154-b5c4-42aa-b305-a719e3254d82" containerName="manager" Jul 01 17:06:30.119309 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.119288 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.134560 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.134531 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d6987486-889cc"] Jul 01 17:06:30.188033 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.187998 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:30.222739 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222699 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tfhcw\" (UniqueName: \"kubernetes.io/projected/ea824a2c-00f7-40ab-96ed-f6d43190c20c-kube-api-access-tfhcw\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.222972 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222757 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.222972 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222785 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-trusted-ca-bundle\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.222972 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222831 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-oauth-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.222972 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222854 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-oauth-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.222972 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222868 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-service-ca\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.223142 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.222970 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.285634 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.285593 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:30.323825 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323767 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323849 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-tfhcw\" (UniqueName: \"kubernetes.io/projected/ea824a2c-00f7-40ab-96ed-f6d43190c20c-kube-api-access-tfhcw\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323891 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323924 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-trusted-ca-bundle\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323950 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-oauth-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.323978 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-oauth-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324010 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.324001 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-service-ca\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324770 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.324742 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324919 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.324760 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-oauth-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324919 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.324844 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-service-ca\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.324919 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.324894 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ea824a2c-00f7-40ab-96ed-f6d43190c20c-trusted-ca-bundle\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.326641 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.326615 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-serving-cert\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.326894 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.326877 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/ea824a2c-00f7-40ab-96ed-f6d43190c20c-console-oauth-config\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.333590 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.333570 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tfhcw\" (UniqueName: \"kubernetes.io/projected/ea824a2c-00f7-40ab-96ed-f6d43190c20c-kube-api-access-tfhcw\") pod \"console-67d6987486-889cc\" (UID: \"ea824a2c-00f7-40ab-96ed-f6d43190c20c\") " pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.428924 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.428836 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:30.565237 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:30.565122 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67d6987486-889cc"] Jul 01 17:06:30.567498 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:06:30.567466 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea824a2c_00f7_40ab_96ed_f6d43190c20c.slice/crio-b7e113f708fa2d9a4216e52344753658be02af4e703447aeed21d9d48e8ee66d WatchSource:0}: Error finding container b7e113f708fa2d9a4216e52344753658be02af4e703447aeed21d9d48e8ee66d: Status 404 returned error can't find the container with id b7e113f708fa2d9a4216e52344753658be02af4e703447aeed21d9d48e8ee66d Jul 01 17:06:31.337291 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:31.337244 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d6987486-889cc" event={"ID":"ea824a2c-00f7-40ab-96ed-f6d43190c20c","Type":"ContainerStarted","Data":"882257ad6b94c534576451393679ec5156ea087120cec9210589dc666bbea2e0"} Jul 01 17:06:31.337826 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:31.337299 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67d6987486-889cc" event={"ID":"ea824a2c-00f7-40ab-96ed-f6d43190c20c","Type":"ContainerStarted","Data":"b7e113f708fa2d9a4216e52344753658be02af4e703447aeed21d9d48e8ee66d"} Jul 01 17:06:31.365761 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:31.365709 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-console/console-67d6987486-889cc" podStartSLOduration=1.365690526 podStartE2EDuration="1.365690526s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:06:30 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:06:31.365160883 +0000 UTC m=+514.604717562" watchObservedRunningTime="2026-07-01 17:06:31.365690526 +0000 UTC m=+514.605247205" Jul 01 17:06:39.243103 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:39.243075 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-84f4454459-s5vgc" Jul 01 17:06:40.336396 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.336360 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:40.367136 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.367100 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-fd65d7d6-crtp5"] Jul 01 17:06:40.371056 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.371032 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.374417 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.374395 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jul 01 17:06:40.374543 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.374420 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-tg4gs\"" Jul 01 17:06:40.388421 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.388393 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-crtp5"] Jul 01 17:06:40.409160 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.409129 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.409318 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.409227 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nds5f\" (UniqueName: \"kubernetes.io/projected/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-kube-api-access-nds5f\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.429358 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.429328 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:40.429522 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.429388 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:40.433993 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.433967 2580 kubelet.go:2654] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:40.473930 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.473892 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:40.509863 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.509829 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nds5f\" (UniqueName: \"kubernetes.io/projected/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-kube-api-access-nds5f\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.510049 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.509982 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.510448 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:06:40.510107 2580 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 01 17:06:40.510448 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:06:40.510205 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs podName:2fab7bac-fac2-4f8f-8ab3-76d338d3f869 nodeName:}" failed. No retries permitted until 2026-07-01 17:06:41.010169408 +0000 UTC m=+524.249726078 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs") pod "model-serving-api-fd65d7d6-crtp5" (UID: "2fab7bac-fac2-4f8f-8ab3-76d338d3f869") : secret "model-serving-api-tls" not found Jul 01 17:06:40.521259 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.521226 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nds5f\" (UniqueName: \"kubernetes.io/projected/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-kube-api-access-nds5f\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:40.874463 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:40.874413 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:41.020771 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.020737 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:41.020963 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:06:41.020917 2580 secret.go:189] Couldn't get secret kserve/model-serving-api-tls: secret "model-serving-api-tls" not found Jul 01 17:06:41.021006 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:06:41.020998 2580 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs podName:2fab7bac-fac2-4f8f-8ab3-76d338d3f869 nodeName:}" failed. No retries permitted until 2026-07-01 17:06:42.020983043 +0000 UTC m=+525.260539703 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "tls-certs" (UniqueName: "kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs") pod "model-serving-api-fd65d7d6-crtp5" (UID: "2fab7bac-fac2-4f8f-8ab3-76d338d3f869") : secret "model-serving-api-tls" not found Jul 01 17:06:41.245674 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.245640 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:41.383956 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.383927 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67d6987486-889cc" Jul 01 17:06:41.442249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.442214 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:41.466194 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.466161 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:06:41.818420 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.818380 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:41.941274 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:41.941238 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:06:42.028485 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:42.028447 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:42.030959 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:42.030930 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/2fab7bac-fac2-4f8f-8ab3-76d338d3f869-tls-certs\") pod \"model-serving-api-fd65d7d6-crtp5\" (UID: \"2fab7bac-fac2-4f8f-8ab3-76d338d3f869\") " pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:42.184673 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:42.184576 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:42.319085 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:42.319057 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-fd65d7d6-crtp5"] Jul 01 17:06:42.320879 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:06:42.320846 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2fab7bac_fac2_4f8f_8ab3_76d338d3f869.slice/crio-081e8b6831d39bf5db9ea09a1825538813e7b8d454231221e60b1747f762a8e2 WatchSource:0}: Error finding container 081e8b6831d39bf5db9ea09a1825538813e7b8d454231221e60b1747f762a8e2: Status 404 returned error can't find the container with id 081e8b6831d39bf5db9ea09a1825538813e7b8d454231221e60b1747f762a8e2 Jul 01 17:06:42.385791 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:42.385754 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-crtp5" event={"ID":"2fab7bac-fac2-4f8f-8ab3-76d338d3f869","Type":"ContainerStarted","Data":"081e8b6831d39bf5db9ea09a1825538813e7b8d454231221e60b1747f762a8e2"} Jul 01 17:06:45.403374 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:45.403287 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-fd65d7d6-crtp5" event={"ID":"2fab7bac-fac2-4f8f-8ab3-76d338d3f869","Type":"ContainerStarted","Data":"60817048eb1d4e1ff8b0b9a54cfdbae2ab0cbf62f955170153d89adf68b69947"} Jul 01 17:06:45.403374 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:45.403361 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:06:45.423020 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:45.422970 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve/model-serving-api-fd65d7d6-crtp5" podStartSLOduration=2.650398092 podStartE2EDuration="5.422953293s" totalImagesPullingTime="2.772555201s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:06:40 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:06:45.42151198 +0000 UTC m=+528.661068657" watchObservedRunningTime="2026-07-01 17:06:45.422953293 +0000 UTC m=+528.662509969" Jul 01 17:06:56.412008 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:06:56.411979 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-fd65d7d6-crtp5" Jul 01 17:07:06.487475 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.487408 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="openshift-console/console-57c9d4678b-zjvcn" podUID="33eba021-7869-4138-babe-81d642f6fcdd" containerName="console" containerID="cri-o://777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7" gracePeriod=15 Jul 01 17:07:06.740027 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.739962 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:07:06.841474 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841437 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841696 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841507 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841696 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841537 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"kube-api-access-798kh\" (UniqueName: \"kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841696 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841565 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841696 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841583 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841944 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841730 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.841944 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841833 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config\") pod \"33eba021-7869-4138-babe-81d642f6fcdd\" (UID: \"33eba021-7869-4138-babe-81d642f6fcdd\") " Jul 01 17:07:06.842086 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.841947 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca" (OuterVolumeSpecName: "service-ca") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:07:06.842224 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.842137 2580 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-service-ca\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.842224 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.842136 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config" (OuterVolumeSpecName: "console-config") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:07:06.842341 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.842181 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:07:06.842394 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.842341 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:07:06.843835 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.843795 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:07:06.843931 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.843828 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:07:06.843931 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.843882 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh" (OuterVolumeSpecName: "kube-api-access-798kh") pod "33eba021-7869-4138-babe-81d642f6fcdd" (UID: "33eba021-7869-4138-babe-81d642f6fcdd"). InnerVolumeSpecName "kube-api-access-798kh". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jul 01 17:07:06.943468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943435 2580 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-oauth-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.943468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943463 2580 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-trusted-ca-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.943468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943472 2580 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/33eba021-7869-4138-babe-81d642f6fcdd-console-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.943468 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943481 2580 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-serving-cert\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.943747 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943490 2580 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-798kh\" (UniqueName: \"kubernetes.io/projected/33eba021-7869-4138-babe-81d642f6fcdd-kube-api-access-798kh\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.943747 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.943498 2580 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/33eba021-7869-4138-babe-81d642f6fcdd-console-oauth-config\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:07:06.969612 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:06.969567 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:07:07.500410 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.500313 2580 generic.go:356] "Generic (PLEG): container finished" podID="33eba021-7869-4138-babe-81d642f6fcdd" containerID="777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7" exitCode=0 Jul 01 17:07:07.500410 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.500362 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-57c9d4678b-zjvcn" Jul 01 17:07:07.500410 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.500400 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9d4678b-zjvcn" event={"ID":"33eba021-7869-4138-babe-81d642f6fcdd","Type":"ContainerDied","Data":"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7"} Jul 01 17:07:07.500970 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.500448 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-57c9d4678b-zjvcn" event={"ID":"33eba021-7869-4138-babe-81d642f6fcdd","Type":"ContainerDied","Data":"6fa605d4310a5d5802fc9503e5f533ebb64fd183bc3e31a82244de67ed20286a"} Jul 01 17:07:07.500970 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.500474 2580 scope.go:117] "RemoveContainer" containerID="777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7" Jul 01 17:07:07.509174 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.509155 2580 scope.go:117] "RemoveContainer" containerID="777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7" Jul 01 17:07:07.509446 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:07:07.509429 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7\": container with ID starting with 777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7 not found: ID does not exist" containerID="777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7" Jul 01 17:07:07.509510 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.509454 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7"} err="failed to get container status \"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7\": rpc error: code = NotFound desc = could not find container \"777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7\": container with ID starting with 777a59115787b91c71af817d176b83180da97b53d1a65d035696bbf437e7f8e7 not found: ID does not exist" Jul 01 17:07:07.519018 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.518989 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:07:07.525183 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:07.525157 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-57c9d4678b-zjvcn"] Jul 01 17:07:09.229272 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:09.229230 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="33eba021-7869-4138-babe-81d642f6fcdd" path="/var/lib/kubelet/pods/33eba021-7869-4138-babe-81d642f6fcdd/volumes" Jul 01 17:07:56.355356 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:56.355320 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:07:56.457198 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:56.457163 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:07:57.177015 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:57.176985 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:07:57.177370 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:07:57.177350 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:08:15.989917 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:08:15.989877 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:08:16.094929 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:08:16.094891 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:08:27.524269 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:08:27.524172 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:08:27.706164 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:08:27.706124 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:10:22.813960 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:10:22.813914 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:10:25.023813 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:10:25.023779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:10:45.676540 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:10:45.676508 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:11:49.997769 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:11:49.997732 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:12.820414 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.820379 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:12.826874 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.826845 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:12.827260 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.827243 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="33eba021-7869-4138-babe-81d642f6fcdd" containerName="console" Jul 01 17:12:12.827353 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.827263 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="33eba021-7869-4138-babe-81d642f6fcdd" containerName="console" Jul 01 17:12:12.827409 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.827362 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="33eba021-7869-4138-babe-81d642f6fcdd" containerName="console" Jul 01 17:12:12.830395 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.830367 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:12.834068 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.834044 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-89809-kube-rbac-proxy-sar-config\"" Jul 01 17:12:12.834068 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.834059 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g2mrd\"" Jul 01 17:12:12.834754 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.834727 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-89809-serving-cert\"" Jul 01 17:12:12.834863 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.834847 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 17:12:12.847581 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.847550 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:12.913414 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.913381 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:12.913586 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.913422 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:12.968422 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:12.968388 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:13.014431 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.014396 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:13.014600 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.014440 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:13.015203 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.015177 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:13.016907 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.016880 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls\") pod \"model-chainer-raw-89809-659f466585-tkfj5\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:13.143149 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.143040 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:13.279782 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.279749 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:13.282929 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:12:13.282900 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc7ff20d2_786b_4828_92e3_4e5058ed083b.slice/crio-3df05d8d1a9ede9a71847d59bd25e4fc7ce3dc225649d1ccd7f8eca2caa9b1aa WatchSource:0}: Error finding container 3df05d8d1a9ede9a71847d59bd25e4fc7ce3dc225649d1ccd7f8eca2caa9b1aa: Status 404 returned error can't find the container with id 3df05d8d1a9ede9a71847d59bd25e4fc7ce3dc225649d1ccd7f8eca2caa9b1aa Jul 01 17:12:13.287982 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.287963 2580 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jul 01 17:12:13.837194 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:13.837156 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" event={"ID":"c7ff20d2-786b-4828-92e3-4e5058ed083b","Type":"ContainerStarted","Data":"3df05d8d1a9ede9a71847d59bd25e4fc7ce3dc225649d1ccd7f8eca2caa9b1aa"} Jul 01 17:12:15.854168 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:15.854128 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" event={"ID":"c7ff20d2-786b-4828-92e3-4e5058ed083b","Type":"ContainerStarted","Data":"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be"} Jul 01 17:12:15.854542 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:15.854266 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:15.874206 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:15.874105 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podStartSLOduration=1.554608509 podStartE2EDuration="3.874086069s" totalImagesPullingTime="2.31947756s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:12:12 +0000 UTC" imagePullSessionsCount=1 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:12:15.871738342 +0000 UTC m=+859.111295020" watchObservedRunningTime="2026-07-01 17:12:15.874086069 +0000 UTC m=+859.113642746" Jul 01 17:12:21.863720 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:21.863690 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:22.857440 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:22.857403 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:22.857687 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:22.857635 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" containerID="cri-o://00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be" gracePeriod=30 Jul 01 17:12:22.895962 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:22.895928 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.045302 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.045269 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.138995 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.138917 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.209782 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.209749 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.293817 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.293779 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.471933 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.471898 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:23.648118 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:23.648084 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:26.862532 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:26.862489 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:31.862569 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:31.862530 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:36.863370 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:36.863329 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:36.863869 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:36.863436 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:37.040334 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:37.040302 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:37.142322 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:37.142237 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:12:41.867105 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:41.867065 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:46.862594 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:46.862551 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:51.863164 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:51.863122 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:12:53.011889 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.011863 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:53.013593 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.013559 2580 generic.go:356] "Generic (PLEG): container finished" podID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerID="00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be" exitCode=0 Jul 01 17:12:53.013711 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.013600 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" Jul 01 17:12:53.013711 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.013608 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" event={"ID":"c7ff20d2-786b-4828-92e3-4e5058ed083b","Type":"ContainerDied","Data":"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be"} Jul 01 17:12:53.013711 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.013661 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5" event={"ID":"c7ff20d2-786b-4828-92e3-4e5058ed083b","Type":"ContainerDied","Data":"3df05d8d1a9ede9a71847d59bd25e4fc7ce3dc225649d1ccd7f8eca2caa9b1aa"} Jul 01 17:12:53.013711 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.013691 2580 scope.go:117] "RemoveContainer" containerID="00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be" Jul 01 17:12:53.021843 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.021797 2580 scope.go:117] "RemoveContainer" containerID="00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be" Jul 01 17:12:53.022144 ip-10-0-138-241 kubenswrapper[2580]: E0701 17:12:53.022120 2580 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be\": container with ID starting with 00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be not found: ID does not exist" containerID="00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be" Jul 01 17:12:53.022196 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.022154 2580 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be"} err="failed to get container status \"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be\": rpc error: code = NotFound desc = could not find container \"00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be\": container with ID starting with 00a4b495dcfe254a9a7a43a8393d160c44cf38f36bf15d4ee309350e251fc8be not found: ID does not exist" Jul 01 17:12:53.147952 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.147859 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls\") pod \"c7ff20d2-786b-4828-92e3-4e5058ed083b\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " Jul 01 17:12:53.147952 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.147930 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle\") pod \"c7ff20d2-786b-4828-92e3-4e5058ed083b\" (UID: \"c7ff20d2-786b-4828-92e3-4e5058ed083b\") " Jul 01 17:12:53.148293 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.148264 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "c7ff20d2-786b-4828-92e3-4e5058ed083b" (UID: "c7ff20d2-786b-4828-92e3-4e5058ed083b"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:12:53.150102 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.150069 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "c7ff20d2-786b-4828-92e3-4e5058ed083b" (UID: "c7ff20d2-786b-4828-92e3-4e5058ed083b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:12:53.248657 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.248621 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/c7ff20d2-786b-4828-92e3-4e5058ed083b-proxy-tls\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:12:53.248657 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.248659 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c7ff20d2-786b-4828-92e3-4e5058ed083b-openshift-service-ca-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:12:53.328738 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.328704 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:53.332874 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:53.332845 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-89809-659f466585-tkfj5"] Jul 01 17:12:55.230746 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:55.230709 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" path="/var/lib/kubelet/pods/c7ff20d2-786b-4828-92e3-4e5058ed083b/volumes" Jul 01 17:12:57.202495 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:57.202463 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:12:57.205433 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:12:57.205410 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:14:03.135167 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.135129 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:03.135628 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.135482 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" Jul 01 17:14:03.135628 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.135493 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" Jul 01 17:14:03.135628 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.135546 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="c7ff20d2-786b-4828-92e3-4e5058ed083b" containerName="model-chainer-raw-89809" Jul 01 17:14:03.138449 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.138425 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.139797 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.139763 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:03.141002 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.140979 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-e08af-kube-rbac-proxy-sar-config\"" Jul 01 17:14:03.141002 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.140996 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jul 01 17:14:03.141298 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.141283 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-g2mrd\"" Jul 01 17:14:03.142216 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.142197 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-e08af-serving-cert\"" Jul 01 17:14:03.149349 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.149327 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:03.227546 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.227508 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.227704 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.227611 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.243767 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.243734 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:03.328632 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.328599 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.328894 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.328865 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.329451 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.329425 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.331107 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.331085 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls\") pod \"model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.449371 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.449326 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:03.582868 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:03.582840 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:03.585051 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:14:03.585023 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49679933_9bc5_423e_9575_476a07122b3b.slice/crio-a69e4aee50c347aa095966a53a227c1b38f17fd2b122c21ac996978ce262eae7 WatchSource:0}: Error finding container a69e4aee50c347aa095966a53a227c1b38f17fd2b122c21ac996978ce262eae7: Status 404 returned error can't find the container with id a69e4aee50c347aa095966a53a227c1b38f17fd2b122c21ac996978ce262eae7 Jul 01 17:14:04.324262 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:04.324224 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" event={"ID":"49679933-9bc5-423e-9575-476a07122b3b","Type":"ContainerStarted","Data":"4b7c9e5ae645cd9823f77affafb2d1c68d6fb1cded7aa85823a07cbf24a46e28"} Jul 01 17:14:04.324698 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:04.324274 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" event={"ID":"49679933-9bc5-423e-9575-476a07122b3b","Type":"ContainerStarted","Data":"a69e4aee50c347aa095966a53a227c1b38f17fd2b122c21ac996978ce262eae7"} Jul 01 17:14:04.324698 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:04.324363 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:04.343161 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:04.343075 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podStartSLOduration=1.343053033 podStartE2EDuration="1.343053033s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=false podCreationTimestamp="2026-07-01 17:14:03 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:14:04.342638933 +0000 UTC m=+967.582195612" watchObservedRunningTime="2026-07-01 17:14:04.343053033 +0000 UTC m=+967.582609710" Jul 01 17:14:10.334260 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:10.334229 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:13.202300 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.202266 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:13.202730 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.202446 2580 kuberuntime_container.go:872] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" containerID="cri-o://4b7c9e5ae645cd9823f77affafb2d1c68d6fb1cded7aa85823a07cbf24a46e28" gracePeriod=30 Jul 01 17:14:13.229872 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.229839 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:13.386225 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.386191 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:13.492863 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.492732 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:13.589023 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.588991 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:13.875641 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:13.875555 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:15.333373 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:15.333329 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:20.332606 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:20.332567 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:25.333254 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:25.333210 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:25.333727 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:25.333355 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:30.333892 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:30.333842 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:33.409974 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:33.409937 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:33.779003 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:33.778966 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:14:35.333365 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:35.333326 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:40.332949 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:40.332900 2580 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jul 01 17:14:43.492957 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.492865 2580 generic.go:356] "Generic (PLEG): container finished" podID="49679933-9bc5-423e-9575-476a07122b3b" containerID="4b7c9e5ae645cd9823f77affafb2d1c68d6fb1cded7aa85823a07cbf24a46e28" exitCode=0 Jul 01 17:14:43.492957 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.492936 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" event={"ID":"49679933-9bc5-423e-9575-476a07122b3b","Type":"ContainerDied","Data":"4b7c9e5ae645cd9823f77affafb2d1c68d6fb1cded7aa85823a07cbf24a46e28"} Jul 01 17:14:43.853765 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.853737 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:43.993072 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.993033 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle\") pod \"49679933-9bc5-423e-9575-476a07122b3b\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " Jul 01 17:14:43.993249 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.993141 2580 reconciler_common.go:163] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls\") pod \"49679933-9bc5-423e-9575-476a07122b3b\" (UID: \"49679933-9bc5-423e-9575-476a07122b3b\") " Jul 01 17:14:43.993463 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.993442 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "49679933-9bc5-423e-9575-476a07122b3b" (UID: "49679933-9bc5-423e-9575-476a07122b3b"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jul 01 17:14:43.995247 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:43.995222 2580 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "49679933-9bc5-423e-9575-476a07122b3b" (UID: "49679933-9bc5-423e-9575-476a07122b3b"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jul 01 17:14:44.094749 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.094665 2580 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/49679933-9bc5-423e-9575-476a07122b3b-proxy-tls\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:14:44.094749 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.094698 2580 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/49679933-9bc5-423e-9575-476a07122b3b-openshift-service-ca-bundle\") on node \"ip-10-0-138-241.ec2.internal\" DevicePath \"\"" Jul 01 17:14:44.498075 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.498036 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" event={"ID":"49679933-9bc5-423e-9575-476a07122b3b","Type":"ContainerDied","Data":"a69e4aee50c347aa095966a53a227c1b38f17fd2b122c21ac996978ce262eae7"} Jul 01 17:14:44.498532 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.498098 2580 scope.go:117] "RemoveContainer" containerID="4b7c9e5ae645cd9823f77affafb2d1c68d6fb1cded7aa85823a07cbf24a46e28" Jul 01 17:14:44.498532 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.498048 2580 util.go:52] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2" Jul 01 17:14:44.522290 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.522255 2580 kubelet.go:2551] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:44.528558 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:44.528533 2580 kubelet.go:2545] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-e08af-6ff4c87b69-xx8d2"] Jul 01 17:14:45.229166 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:14:45.229137 2580 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="49679933-9bc5-423e-9575-476a07122b3b" path="/var/lib/kubelet/pods/49679933-9bc5-423e-9575-476a07122b3b/volumes" Jul 01 17:16:08.792456 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:16:08.792415 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:16:10.291569 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:16:10.291517 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:16:36.149719 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:16:36.149682 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:17:57.229356 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:17:57.229326 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:17:57.233456 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:17:57.233431 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:18:18.928797 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:18:18.928764 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:18:58.956101 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:18:58.956064 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:19:03.997483 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:19:03.997442 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:19:04.442767 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:19:04.442726 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:19:27.510766 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:19:27.510727 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:34.222065 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:34.221961 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:34.400743 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:34.400709 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:48.316494 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:48.316457 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:48.390537 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:48.390493 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:53.470714 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:53.470676 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:20:53.936105 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:20:53.936007 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:21:03.520959 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:21:03.520916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:21:08.674277 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:21:08.674240 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:21:09.673390 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:21:09.673353 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:22:39.014384 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:22:39.014352 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:22:44.053196 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:22:44.053152 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:22:44.327677 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:22:44.327592 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:22:57.255005 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:22:57.254975 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:22:57.258900 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:22:57.258872 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:24:05.130340 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:05.130307 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:09.811540 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:09.811503 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:09.909713 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:09.909673 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:40.258864 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:40.258788 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:40.386897 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:40.386869 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:40.485351 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:40.485300 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:43.758185 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:43.758147 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-7tknz_ffc63e3c-606c-44ad-abd1-d8c725769df3/global-pull-secret-syncer/0.log" Jul 01 17:24:43.887030 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:43.886992 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" Jul 01 17:24:43.997783 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:43.997744 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-138-241.ec2.internal_400088389ad348a7c1f16971b84768a2/haproxy/0.log" Jul 01 17:24:46.068682 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:46.068646 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:46.152950 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:46.152916 2580 log.go:25] "Finished parsing log file, hit bytes limit" path="/var/log/pods/kube-system_konnectivity-agent-4pjmt_b7b7e371-fbff-4505-897b-610cc43b146f/konnectivity-agent/0.log" limit=1024 Jul 01 17:24:48.154684 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.154654 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-65448c9fc7-dtrb2_4676fb6a-72b0-4cb2-b053-133b2c4ded9c/metrics-server/0.log" Jul 01 17:24:48.410299 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.410194 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/node-exporter/0.log" Jul 01 17:24:48.437676 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.437647 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/kube-rbac-proxy/0.log" Jul 01 17:24:48.463784 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.463758 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-lnrxx_fc1ca859-c7ec-48fe-81ee-107c9a04bf55/init-textfile/0.log" Jul 01 17:24:48.770721 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.770679 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-gxdxh_f4869407-5981-463b-9f9e-5ca9599b3337/prometheus-operator/0.log" Jul 01 17:24:48.806221 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.806187 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-7b7d896497-gxdxh_f4869407-5981-463b-9f9e-5ca9599b3337/kube-rbac-proxy/0.log" Jul 01 17:24:48.890481 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.890450 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5ccc6dd5-4kmf7_375451d5-4506-4a94-af88-7e967014b1a4/telemeter-client/0.log" Jul 01 17:24:48.922952 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.922923 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5ccc6dd5-4kmf7_375451d5-4506-4a94-af88-7e967014b1a4/reload/0.log" Jul 01 17:24:48.949069 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.949041 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-6f5ccc6dd5-4kmf7_375451d5-4506-4a94-af88-7e967014b1a4/kube-rbac-proxy/0.log" Jul 01 17:24:48.985229 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:48.985197 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/thanos-query/0.log" Jul 01 17:24:49.021054 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:49.020977 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-web/0.log" Jul 01 17:24:49.043761 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:49.043735 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy/0.log" Jul 01 17:24:49.066591 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:49.066562 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/prom-label-proxy/0.log" Jul 01 17:24:49.091440 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:49.091414 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-rules/0.log" Jul 01 17:24:49.126865 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:49.126832 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-69995699bc-tqnfz_db99495f-b0ab-4c67-a198-87461a7f7bb4/kube-rbac-proxy-metrics/0.log" Jul 01 17:24:51.014245 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:51.014214 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/2.log" Jul 01 17:24:51.018032 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:51.017997 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-7fdbdf9c99-dcvzq_786e68e3-1a0e-4107-80c2-0b908527ac1f/console-operator/3.log" Jul 01 17:24:51.444768 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:51.444741 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67d6987486-889cc_ea824a2c-00f7-40ab-96ed-f6d43190c20c/console/0.log" Jul 01 17:24:51.480080 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:51.480044 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-5499b6fb8f-9sghz_0f1d0bc2-7b05-4877-82e5-e176ebe68fc9/download-server/0.log" Jul 01 17:24:52.209379 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.209333 2580 kubelet.go:2535] "SyncLoop ADD" source="api" pods=["openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw"] Jul 01 17:24:52.209761 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.209672 2580 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" Jul 01 17:24:52.209761 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.209684 2580 state_mem.go:107] "Deleted CPUSet assignment" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" Jul 01 17:24:52.209761 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.209745 2580 memory_manager.go:368] "RemoveStaleState removing state" podUID="49679933-9bc5-423e-9575-476a07122b3b" containerName="model-chainer-raw-hpa-e08af" Jul 01 17:24:52.212752 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.212731 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.215137 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.215113 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crfk4\"/\"openshift-service-ca.crt\"" Jul 01 17:24:52.215257 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.215192 2580 reflector.go:439] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-crfk4\"/\"kube-root-ca.crt\"" Jul 01 17:24:52.216266 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.216247 2580 reflector.go:439] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-crfk4\"/\"default-dockercfg-s5btw\"" Jul 01 17:24:52.222724 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.222696 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw"] Jul 01 17:24:52.226389 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.226365 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-proc\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.226497 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.226401 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-lib-modules\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.226497 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.226434 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-sys\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.226576 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.226553 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrng\" (UniqueName: \"kubernetes.io/projected/14c1e04d-9422-4385-b962-7184e85f28bf-kube-api-access-nqrng\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.226614 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.226601 2580 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-podres\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.327752 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327718 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-podres\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.327752 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327759 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-proc\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327776 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-lib-modules\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327798 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-sys\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327872 2580 reconciler_common.go:225] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrng\" (UniqueName: \"kubernetes.io/projected/14c1e04d-9422-4385-b962-7184e85f28bf-kube-api-access-nqrng\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-lib-modules\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327947 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-sys\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328042 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.327987 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-proc\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.328250 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.328046 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/14c1e04d-9422-4385-b962-7184e85f28bf-podres\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.337068 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.337033 2580 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrng\" (UniqueName: \"kubernetes.io/projected/14c1e04d-9422-4385-b962-7184e85f28bf-kube-api-access-nqrng\") pod \"perf-node-gather-daemonset-j5zpw\" (UID: \"14c1e04d-9422-4385-b962-7184e85f28bf\") " pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.523022 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.522923 2580 util.go:34] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:52.656616 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:52.656591 2580 kubelet.go:2542] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw"] Jul 01 17:24:52.658839 ip-10-0-138-241 kubenswrapper[2580]: W0701 17:24:52.658795 2580 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod14c1e04d_9422_4385_b962_7184e85f28bf.slice/crio-1f8280a5b9d33201db0498f6089205a06b73fb9eac00ae30f29ad0561872cb50 WatchSource:0}: Error finding container 1f8280a5b9d33201db0498f6089205a06b73fb9eac00ae30f29ad0561872cb50: Status 404 returned error can't find the container with id 1f8280a5b9d33201db0498f6089205a06b73fb9eac00ae30f29ad0561872cb50 Jul 01 17:24:53.020309 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.020276 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4jv9_e6d11153-a3e0-4eb0-b679-3448bc2c4ab1/dns/0.log" Jul 01 17:24:53.043932 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.043903 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-p4jv9_e6d11153-a3e0-4eb0-b679-3448bc2c4ab1/kube-rbac-proxy/0.log" Jul 01 17:24:53.117858 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.117799 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-hgs2r_f3e0f1c8-be07-4739-844d-7d02b51b42cc/dns-node-resolver/0.log" Jul 01 17:24:53.126866 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.126839 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" event={"ID":"14c1e04d-9422-4385-b962-7184e85f28bf","Type":"ContainerStarted","Data":"a291c4e25ce1bf9d481b3c45418018d6c300604cfca8bbbedecc0a8a33c7a9c8"} Jul 01 17:24:53.127034 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.126885 2580 kubelet.go:2566] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" event={"ID":"14c1e04d-9422-4385-b962-7184e85f28bf","Type":"ContainerStarted","Data":"1f8280a5b9d33201db0498f6089205a06b73fb9eac00ae30f29ad0561872cb50"} Jul 01 17:24:53.127034 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.126978 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:24:53.147033 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.146981 2580 pod_startup_latency_tracker.go:144] "Observed pod startup duration" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" podStartSLOduration=1.146963878 podStartE2EDuration="1.146963878s" totalImagesPullingTime="0s" totalInitContainerRuntime="0s" isStatefulPod=true podCreationTimestamp="2026-07-01 17:24:52 +0000 UTC" imagePullSessionsCount=0 imagePullSessionsStartsCount=0 observedRunningTime="2026-07-01 17:24:53.14582082 +0000 UTC m=+1616.385377496" watchObservedRunningTime="2026-07-01 17:24:53.146963878 +0000 UTC m=+1616.386520556" Jul 01 17:24:53.784253 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:53.784224 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-922gs_13204f90-24a2-4ee1-83c1-e6a31a5a76b8/node-ca/0.log" Jul 01 17:24:55.113925 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:55.113900 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-b8tlp_fdcb703e-3e60-4b6f-a86a-9bf94608a44f/serve-healthcheck-canary/0.log" Jul 01 17:24:55.649907 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:55.649878 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq86c_513ed43d-ab12-4a4c-a9c7-bd939d193470/kube-rbac-proxy/0.log" Jul 01 17:24:55.688676 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:55.688645 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq86c_513ed43d-ab12-4a4c-a9c7-bd939d193470/exporter/0.log" Jul 01 17:24:55.715758 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:55.715725 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-lq86c_513ed43d-ab12-4a4c-a9c7-bd939d193470/extractor/0.log" Jul 01 17:24:57.885418 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:57.885381 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-84f4454459-s5vgc_1a0b5809-49a0-40e3-864b-014af014bd39/manager/0.log" Jul 01 17:24:57.935008 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:57.934982 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-fd65d7d6-crtp5_2fab7bac-fac2-4f8f-8ab3-76d338d3f869/server/0.log" Jul 01 17:24:59.141654 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:24:59.141626 2580 kubelet.go:2654] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-crfk4/perf-node-gather-daemonset-j5zpw" Jul 01 17:25:03.929583 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:03.929508 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/kube-multus-additional-cni-plugins/0.log" Jul 01 17:25:03.951999 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:03.951971 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/egress-router-binary-copy/0.log" Jul 01 17:25:03.976202 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:03.976174 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/cni-plugins/0.log" Jul 01 17:25:04.002778 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.002746 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/bond-cni-plugin/0.log" Jul 01 17:25:04.028095 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.028066 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/routeoverride-cni/0.log" Jul 01 17:25:04.050716 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.050659 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/whereabouts-cni-bincopy/0.log" Jul 01 17:25:04.075617 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.075587 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-k8sxt_c4909179-8cdd-41ef-b211-0915ff19bc87/whereabouts-cni/0.log" Jul 01 17:25:04.330388 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.330353 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-d2bhk_40c69be1-a415-4029-b534-1ec9ea957fef/kube-multus/0.log" Jul 01 17:25:04.467343 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.467312 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tz2kn_29b0fcf2-fa42-408f-9310-97306bbfb0de/network-metrics-daemon/0.log" Jul 01 17:25:04.493247 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:04.493220 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-tz2kn_29b0fcf2-fa42-408f-9310-97306bbfb0de/kube-rbac-proxy/0.log" Jul 01 17:25:05.484520 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.484488 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/ovn-controller/0.log" Jul 01 17:25:05.502117 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.502087 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/ovn-acl-logging/0.log" Jul 01 17:25:05.529262 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.529232 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/kube-rbac-proxy-node/0.log" Jul 01 17:25:05.556941 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.556916 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/kube-rbac-proxy-ovn-metrics/0.log" Jul 01 17:25:05.581518 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.581487 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/northd/0.log" Jul 01 17:25:05.604067 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.604038 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/nbdb/0.log" Jul 01 17:25:05.628142 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.628116 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/sbdb/0.log" Jul 01 17:25:05.678698 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:05.678659 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-g8mk4_9ded3f35-ecd0-4c5f-8873-7ed0743a66d9/ovnkube-controller/0.log" Jul 01 17:25:06.886655 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:06.886629 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-source-56499946bc-bf599_c58b2c02-41e8-4f2c-945b-24116210d6f0/check-endpoints/0.log" Jul 01 17:25:06.942530 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:06.942500 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-vncqb_a4003675-8d98-49fd-8ea9-7d8633600257/network-check-target-container/0.log" Jul 01 17:25:08.065618 ip-10-0-138-241 kubenswrapper[2580]: I0701 17:25:08.065592 2580 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-ngvm6_b9e41189-1f28-451e-ad86-f1e7c420c40c/iptables-alerter/0.log"