Jun 02 15:43:13.259522 ip-10-0-128-129 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 15:43:13.259537 ip-10-0-128-129 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 15:43:13.259546 ip-10-0-128-129 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 15:43:13.259878 ip-10-0-128-129 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 15:43:23.348947 ip-10-0-128-129 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 15:43:23.348964 ip-10-0-128-129 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot 990c7f48ebf5424ca5fefcd4d6226e0f -- Jun 02 15:45:50.615532 ip-10-0-128-129 systemd[1]: Starting Kubernetes Kubelet... Jun 02 15:45:51.027285 ip-10-0-128-129 kubenswrapper[2581]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:45:51.027285 ip-10-0-128-129 kubenswrapper[2581]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 15:45:51.027285 ip-10-0-128-129 kubenswrapper[2581]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:45:51.027285 ip-10-0-128-129 kubenswrapper[2581]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 15:45:51.027285 ip-10-0-128-129 kubenswrapper[2581]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 15:45:51.028721 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.028630 2581 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033295 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033318 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033322 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033325 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033328 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:45:51.033320 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033330 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033333 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033336 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033339 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033342 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033344 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033347 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033351 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033356 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033359 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033362 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033364 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033367 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033370 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033372 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033375 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033377 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033379 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033383 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:45:51.033534 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033388 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033391 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033394 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033398 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033402 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033406 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033410 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033412 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033415 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033418 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033420 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033424 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033426 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033429 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033431 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033435 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033438 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033441 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033444 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:45:51.033987 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033447 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033449 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033452 2581 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033454 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033457 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033460 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033462 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033465 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033468 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033471 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033473 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033482 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033485 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033487 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033490 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033492 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033495 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033498 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033501 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033503 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:45:51.034535 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033506 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033508 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033511 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033513 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033516 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033518 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033521 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033525 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033528 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033531 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033533 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033536 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033539 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033542 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033544 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033548 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033551 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033553 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033556 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033558 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:45:51.035006 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033561 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033563 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033566 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033959 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033964 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033968 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033971 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033973 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033976 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033978 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033981 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033983 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033986 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033988 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033991 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033994 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.033997 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034001 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034004 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:45:51.035511 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034007 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034010 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034013 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034016 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034018 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034021 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034023 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034026 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034029 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034031 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034034 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034036 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034039 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034042 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034044 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034046 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034049 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034051 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034053 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034056 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:45:51.035955 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034059 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034061 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034063 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034066 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034069 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034071 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034074 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034076 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034078 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034081 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034083 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034085 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034089 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034096 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034100 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034103 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034106 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034110 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034113 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034115 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:45:51.036503 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034118 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034121 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034124 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034127 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034130 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034132 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034135 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034138 2581 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034140 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034143 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034145 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034147 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034150 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034152 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034155 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034157 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034159 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034162 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034164 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:45:51.036996 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034167 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034169 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034171 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034173 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034176 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034178 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034183 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034185 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034188 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034191 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034193 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034285 2581 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034293 2581 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034299 2581 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034303 2581 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034308 2581 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034312 2581 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034316 2581 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034321 2581 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034324 2581 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034327 2581 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 15:45:51.037484 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034330 2581 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034333 2581 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034336 2581 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034339 2581 flags.go:64] FLAG: --cgroup-root="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034342 2581 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034345 2581 flags.go:64] FLAG: --client-ca-file="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034348 2581 flags.go:64] FLAG: --cloud-config="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034350 2581 flags.go:64] FLAG: --cloud-provider="external" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034353 2581 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034357 2581 flags.go:64] FLAG: --cluster-domain="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034360 2581 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034363 2581 flags.go:64] FLAG: --config-dir="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034366 2581 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034369 2581 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034373 2581 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034376 2581 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034380 2581 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034384 2581 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034387 2581 flags.go:64] FLAG: --contention-profiling="false" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034390 2581 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034393 2581 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034396 2581 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034399 2581 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034403 2581 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034406 2581 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 15:45:51.038003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034409 2581 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034411 2581 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034415 2581 flags.go:64] FLAG: --enable-server="true" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034418 2581 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034422 2581 flags.go:64] FLAG: --event-burst="100" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034425 2581 flags.go:64] FLAG: --event-qps="50" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034428 2581 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034431 2581 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034434 2581 flags.go:64] FLAG: --eviction-hard="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034437 2581 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034440 2581 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034443 2581 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034446 2581 flags.go:64] FLAG: --eviction-soft="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034448 2581 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034451 2581 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034454 2581 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034457 2581 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034460 2581 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034463 2581 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034465 2581 flags.go:64] FLAG: --feature-gates="" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034469 2581 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034472 2581 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034475 2581 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034478 2581 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034483 2581 flags.go:64] FLAG: --healthz-port="10248" Jun 02 15:45:51.039030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034486 2581 flags.go:64] FLAG: --help="false" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034489 2581 flags.go:64] FLAG: --hostname-override="ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034492 2581 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034495 2581 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034498 2581 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034501 2581 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034504 2581 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034507 2581 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034510 2581 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034512 2581 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034516 2581 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034519 2581 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034522 2581 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034525 2581 flags.go:64] FLAG: --kube-reserved="" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034527 2581 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034530 2581 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034533 2581 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034535 2581 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034538 2581 flags.go:64] FLAG: --lock-file="" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034541 2581 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034544 2581 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034547 2581 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034552 2581 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 15:45:51.039651 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034555 2581 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034558 2581 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034561 2581 flags.go:64] FLAG: --logging-format="text" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034564 2581 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034567 2581 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034570 2581 flags.go:64] FLAG: --manifest-url="" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034572 2581 flags.go:64] FLAG: --manifest-url-header="" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034576 2581 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034583 2581 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034587 2581 flags.go:64] FLAG: --max-pods="110" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034590 2581 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034593 2581 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034596 2581 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034598 2581 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034601 2581 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034604 2581 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034607 2581 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034614 2581 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034617 2581 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034619 2581 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034625 2581 flags.go:64] FLAG: --pod-cidr="" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034628 2581 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034634 2581 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034637 2581 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 15:45:51.040212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034640 2581 flags.go:64] FLAG: --pods-per-core="0" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034643 2581 flags.go:64] FLAG: --port="10250" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034646 2581 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034649 2581 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-092ce39c1a60e525d" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034652 2581 flags.go:64] FLAG: --qos-reserved="" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034655 2581 flags.go:64] FLAG: --read-only-port="10255" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034658 2581 flags.go:64] FLAG: --register-node="true" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034661 2581 flags.go:64] FLAG: --register-schedulable="true" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034663 2581 flags.go:64] FLAG: --register-with-taints="" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034667 2581 flags.go:64] FLAG: --registry-burst="10" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034669 2581 flags.go:64] FLAG: --registry-qps="5" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034672 2581 flags.go:64] FLAG: --reserved-cpus="" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034675 2581 flags.go:64] FLAG: --reserved-memory="" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034678 2581 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034681 2581 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034684 2581 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034688 2581 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034693 2581 flags.go:64] FLAG: --runonce="false" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034696 2581 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034699 2581 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034702 2581 flags.go:64] FLAG: --seccomp-default="false" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034704 2581 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034707 2581 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034710 2581 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034713 2581 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034716 2581 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 15:45:51.040805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034718 2581 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034721 2581 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034724 2581 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034727 2581 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034730 2581 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034733 2581 flags.go:64] FLAG: --system-cgroups="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034736 2581 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034741 2581 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034744 2581 flags.go:64] FLAG: --tls-cert-file="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034747 2581 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034751 2581 flags.go:64] FLAG: --tls-min-version="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034754 2581 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034756 2581 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034759 2581 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034762 2581 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034764 2581 flags.go:64] FLAG: --v="2" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034768 2581 flags.go:64] FLAG: --version="false" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034772 2581 flags.go:64] FLAG: --vmodule="" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034776 2581 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.034779 2581 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034889 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034893 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034897 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034901 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:45:51.041477 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034904 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034907 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034910 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034913 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034916 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034918 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034921 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034924 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034927 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034930 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034932 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034935 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034938 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034940 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034943 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034945 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034948 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034950 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034952 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034955 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:45:51.042092 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034957 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034960 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034964 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034968 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034971 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034973 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034976 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034978 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034980 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034983 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034987 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034991 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034993 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034996 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.034998 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035001 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035004 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035006 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035009 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035011 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:45:51.042610 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035014 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035017 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035020 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035022 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035024 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035028 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035030 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035033 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035035 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035038 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035040 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035043 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035045 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035047 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035050 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035052 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035055 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035057 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035059 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035062 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:45:51.043127 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035064 2581 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035067 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035070 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035074 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035076 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035079 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035081 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035083 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035087 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035091 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035093 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035096 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035099 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035102 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035104 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035107 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035110 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035112 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035115 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:45:51.043646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035118 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035120 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.035123 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.035841 2581 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.042168 2581 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.042186 2581 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042234 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042239 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042242 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042245 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042248 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042251 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042254 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042257 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042259 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042275 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:45:51.044105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042279 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042281 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042284 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042286 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042289 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042292 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042294 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042297 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042299 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042302 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042304 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042307 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042309 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042312 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042314 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042317 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042320 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042322 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042325 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042327 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:45:51.044562 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042329 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042333 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042336 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042338 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042341 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042343 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042346 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042348 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042351 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042353 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042356 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042358 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042360 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042363 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042366 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042369 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042371 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042374 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042377 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:45:51.045054 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042379 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042382 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042384 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042387 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042389 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042393 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042397 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042400 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042404 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042406 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042409 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042412 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042414 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042417 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042420 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042424 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042426 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042429 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042431 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:45:51.045568 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042433 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042436 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042439 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042441 2581 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042443 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042447 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042449 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042452 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042455 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042457 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042460 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042462 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042464 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042467 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042469 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042472 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042476 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:45:51.046073 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042480 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.042485 2581 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042583 2581 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042587 2581 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042590 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042593 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042596 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042598 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042601 2581 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042603 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042606 2581 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042609 2581 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042612 2581 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042615 2581 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042618 2581 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042620 2581 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 15:45:51.046514 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042623 2581 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042625 2581 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042628 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042631 2581 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042633 2581 feature_gate.go:328] unrecognized feature gate: Example Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042636 2581 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042638 2581 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042641 2581 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042643 2581 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042646 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042648 2581 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042651 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042653 2581 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042656 2581 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042658 2581 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042660 2581 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042663 2581 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042665 2581 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042667 2581 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042670 2581 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 15:45:51.046905 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042672 2581 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042674 2581 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042677 2581 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042679 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042681 2581 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042684 2581 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042687 2581 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042690 2581 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042692 2581 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042695 2581 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042697 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042700 2581 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042702 2581 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042704 2581 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042707 2581 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042709 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042712 2581 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042714 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042717 2581 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 15:45:51.047407 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042719 2581 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042721 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042724 2581 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042726 2581 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042728 2581 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042732 2581 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042735 2581 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042738 2581 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042741 2581 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042743 2581 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042747 2581 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042750 2581 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042752 2581 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042755 2581 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042757 2581 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042760 2581 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042762 2581 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042764 2581 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 15:45:51.047861 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042767 2581 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042769 2581 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042772 2581 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042774 2581 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042777 2581 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042780 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042783 2581 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042785 2581 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042787 2581 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042790 2581 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042792 2581 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042794 2581 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042797 2581 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042799 2581 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:51.042801 2581 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.042806 2581 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 15:45:51.048307 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.043508 2581 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 15:45:51.048727 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.045394 2581 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 15:45:51.048727 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.046305 2581 server.go:1019] "Starting client certificate rotation" Jun 02 15:45:51.048727 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.046399 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 15:45:51.048727 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.047089 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 15:45:51.069437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.069413 2581 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 15:45:51.070617 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.070601 2581 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 15:45:51.085013 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.084988 2581 log.go:25] "Validated CRI v1 runtime API" Jun 02 15:45:51.090082 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.090067 2581 log.go:25] "Validated CRI v1 image API" Jun 02 15:45:51.093424 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.093404 2581 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 15:45:51.097098 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.097079 2581 fs.go:135] Filesystem UUIDs: map[25469c1d-b8f9-4708-a4b0-872b66277309:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 8d2fd601-8f75-4230-b754-f90ecefc1ae1:/dev/nvme0n1p4] Jun 02 15:45:51.097146 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.097099 2581 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 15:45:51.101863 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.101747 2581 manager.go:217] Machine: {Timestamp:2026-06-02 15:45:51.101204848 +0000 UTC m=+0.375636905 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:2926952 MemoryCapacity:32812175360 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2faa03173a826d94090209e451cbf5 SystemUUID:ec2faa03-173a-826d-9409-0209e451cbf5 BootID:990c7f48-ebf5-424c-a5fe-fcd4d6226e0f Filesystems:[{Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406085632 Type:vfs Inodes:4005392 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406089728 Type:vfs Inodes:1048576 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4e:69:c0:24:4f Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4e:69:c0:24:4f Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ba:62:37:01:05:ac Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812175360 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 15:45:51.102557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.102545 2581 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 15:45:51.102647 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.102634 2581 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 15:45:51.104323 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.104290 2581 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 15:45:51.104463 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.104327 2581 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-129.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 15:45:51.104508 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.104474 2581 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 15:45:51.104508 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.104484 2581 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 15:45:51.104508 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.104491 2581 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 15:45:51.105076 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.105067 2581 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 15:45:51.106245 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.106235 2581 state_mem.go:36] "Initialized new in-memory state store" Jun 02 15:45:51.106560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.106550 2581 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 15:45:51.108780 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108769 2581 kubelet.go:491] "Attempting to sync node with API server" Jun 02 15:45:51.108820 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108793 2581 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 15:45:51.108820 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108810 2581 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 15:45:51.108874 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108823 2581 kubelet.go:397] "Adding apiserver pod source" Jun 02 15:45:51.108874 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108837 2581 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 15:45:51.108999 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.108982 2581 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 15:45:51.110317 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.110306 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 15:45:51.110367 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.110324 2581 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 15:45:51.113164 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.113141 2581 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 15:45:51.114867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.114854 2581 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 15:45:51.117295 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117255 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 15:45:51.117338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117305 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 15:45:51.117338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117313 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 15:45:51.117338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117318 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 15:45:51.117338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117328 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 15:45:51.117338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117335 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117341 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117346 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117354 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117359 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117368 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 15:45:51.117474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.117373 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 15:45:51.118282 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.118273 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 15:45:51.118318 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.118282 2581 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 15:45:51.121618 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.121605 2581 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 15:45:51.121693 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.121648 2581 server.go:1295] "Started kubelet" Jun 02 15:45:51.121750 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.121721 2581 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 15:45:51.121784 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.121731 2581 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 15:45:51.121829 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.121817 2581 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 15:45:51.122505 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.122472 2581 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-129.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 15:45:51.122619 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.122565 2581 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 15:45:51.122679 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.122649 2581 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 15:45:51.122625 ip-10-0-128-129 systemd[1]: Started Kubernetes Kubelet. Jun 02 15:45:51.122950 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.122930 2581 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 15:45:51.123251 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.123212 2581 server.go:317] "Adding debug handlers to kubelet server" Jun 02 15:45:51.128096 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.128077 2581 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 15:45:51.128815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.128798 2581 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 15:45:51.129043 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.129021 2581 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 15:45:51.129043 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.129043 2581 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 15:45:51.129196 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.129183 2581 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 15:45:51.129253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.129232 2581 reconstruct.go:97] "Volume reconstruction finished" Jun 02 15:45:51.129253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.129246 2581 reconciler.go:26] "Reconciler: start to sync state" Jun 02 15:45:51.131167 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.130900 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.131646 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.131606 2581 factory.go:55] Registering systemd factory Jun 02 15:45:51.131646 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.131623 2581 factory.go:223] Registration of the systemd container factory successfully Jun 02 15:45:51.131928 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.131765 2581 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-128-129.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 15:45:51.132124 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.132101 2581 factory.go:153] Registering CRI-O factory Jun 02 15:45:51.132124 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.132121 2581 factory.go:223] Registration of the crio container factory successfully Jun 02 15:45:51.132212 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.132090 2581 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 15:45:51.132212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.132207 2581 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 15:45:51.132293 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.132232 2581 factory.go:103] Registering Raw factory Jun 02 15:45:51.132293 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.132248 2581 manager.go:1196] Started watching for new ooms in manager Jun 02 15:45:51.132867 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.128306 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-129.ec2.internal.18b54e259b53d451 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-129.ec2.internal,UID:ip-10-0-128-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-129.ec2.internal,},FirstTimestamp:2026-06-02 15:45:51.121618001 +0000 UTC m=+0.396050056,LastTimestamp:2026-06-02 15:45:51.121618001 +0000 UTC m=+0.396050056,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-129.ec2.internal,}" Jun 02 15:45:51.133438 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.133422 2581 manager.go:319] Starting recovery of all containers Jun 02 15:45:51.134670 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.134641 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crzgk" Jun 02 15:45:51.135055 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.135035 2581 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 15:45:51.142559 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.142438 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-crzgk" Jun 02 15:45:51.144211 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.144195 2581 manager.go:324] Recovery completed Jun 02 15:45:51.148828 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.148812 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.150862 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.150843 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.150945 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.150872 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.150945 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.150884 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.151436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.151420 2581 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 15:45:51.151436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.151435 2581 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 15:45:51.151561 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.151452 2581 state_mem.go:36] "Initialized new in-memory state store" Jun 02 15:45:51.153202 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.153137 2581 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-129.ec2.internal.18b54e259d12046a default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-129.ec2.internal,UID:ip-10-0-128-129.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:NodeHasSufficientMemory,Message:Node ip-10-0-128-129.ec2.internal status is now: NodeHasSufficientMemory,Source:EventSource{Component:kubelet,Host:ip-10-0-128-129.ec2.internal,},FirstTimestamp:2026-06-02 15:45:51.15085937 +0000 UTC m=+0.425291425,LastTimestamp:2026-06-02 15:45:51.15085937 +0000 UTC m=+0.425291425,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-129.ec2.internal,}" Jun 02 15:45:51.153508 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.153483 2581 policy_none.go:49] "None policy: Start" Jun 02 15:45:51.153570 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.153511 2581 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 15:45:51.153570 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.153522 2581 state_mem.go:35] "Initializing new in-memory state store" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.188829 2581 manager.go:341] "Starting Device Plugin manager" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.188868 2581 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.188877 2581 server.go:85] "Starting device plugin registration server" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.189134 2581 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.189147 2581 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.189222 2581 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.189320 2581 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.189329 2581 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.189935 2581 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 15:45:51.194405 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.189970 2581 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.250256 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.250204 2581 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 15:45:51.251745 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.251721 2581 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 15:45:51.251834 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.251754 2581 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 15:45:51.251834 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.251781 2581 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 15:45:51.251834 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.251790 2581 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 15:45:51.251963 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.251837 2581 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 15:45:51.255821 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.255793 2581 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:45:51.289585 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.289515 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.291257 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.291240 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.291351 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.291283 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.291351 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.291293 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.291351 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.291316 2581 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.301847 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.301826 2581 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.301931 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.301848 2581 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-129.ec2.internal\": node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.323980 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.323961 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.352906 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.352858 2581 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal"] Jun 02 15:45:51.353026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.352949 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.353841 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.353823 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.353907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.353855 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.353907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.353865 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.354979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.354967 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.355113 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355101 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.355150 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355130 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.355633 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355618 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.355718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355619 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.355718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355666 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.355718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355677 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.355718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355644 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.355925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.355727 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.356713 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.356698 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.356762 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.356726 2581 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 15:45:51.357391 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.357376 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientMemory" Jun 02 15:45:51.357467 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.357401 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 15:45:51.357467 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.357413 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeHasSufficientPID" Jun 02 15:45:51.382126 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.382097 2581 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-129.ec2.internal\" not found" node="ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.385384 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.385367 2581 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-129.ec2.internal\" not found" node="ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.424509 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.424477 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.431849 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.431822 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.431918 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.431854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.431918 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.431876 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e0a2c8ef673f3324a54cfd46ade1a9c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-129.ec2.internal\" (UID: \"7e0a2c8ef673f3324a54cfd46ade1a9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.525558 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.525530 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.532935 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.532914 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.532994 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.532942 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.532994 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.532960 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e0a2c8ef673f3324a54cfd46ade1a9c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-129.ec2.internal\" (UID: \"7e0a2c8ef673f3324a54cfd46ade1a9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.533067 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.533016 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.533067 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.533033 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/0b2b7bba8dc600d4bbe83acf7ebb93a3-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal\" (UID: \"0b2b7bba8dc600d4bbe83acf7ebb93a3\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.533067 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.533007 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/7e0a2c8ef673f3324a54cfd46ade1a9c-config\") pod \"kube-apiserver-proxy-ip-10-0-128-129.ec2.internal\" (UID: \"7e0a2c8ef673f3324a54cfd46ade1a9c\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.625969 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.625889 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.684446 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.684422 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.688899 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:51.688879 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:51.726342 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.726314 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.826911 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.826879 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:51.927467 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:51.927440 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.028096 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:52.028067 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.046614 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.046591 2581 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 15:45:52.046742 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.046728 2581 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 15:45:52.128170 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:52.128135 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.128170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.128158 2581 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 15:45:52.144035 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.143998 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 15:40:51 +0000 UTC" deadline="2027-11-01 23:05:06.056833082 +0000 UTC" Jun 02 15:45:52.144035 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.144025 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12415h19m13.912811308s" Jun 02 15:45:52.149805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.149778 2581 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 15:45:52.170753 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.170725 2581 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4snm" Jun 02 15:45:52.177576 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.177428 2581 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-x4snm" Jun 02 15:45:52.187719 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:52.187685 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0b2b7bba8dc600d4bbe83acf7ebb93a3.slice/crio-20e84dfc0d9407a928a6496880cad57d0ae78fdd0b4c2570ef32ebc71034c127 WatchSource:0}: Error finding container 20e84dfc0d9407a928a6496880cad57d0ae78fdd0b4c2570ef32ebc71034c127: Status 404 returned error can't find the container with id 20e84dfc0d9407a928a6496880cad57d0ae78fdd0b4c2570ef32ebc71034c127 Jun 02 15:45:52.187933 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:52.187911 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7e0a2c8ef673f3324a54cfd46ade1a9c.slice/crio-49476b132dd5abe32a4680724d0dd58ddf93f8a81e6e7eb7a19f0a06df66b4ae WatchSource:0}: Error finding container 49476b132dd5abe32a4680724d0dd58ddf93f8a81e6e7eb7a19f0a06df66b4ae: Status 404 returned error can't find the container with id 49476b132dd5abe32a4680724d0dd58ddf93f8a81e6e7eb7a19f0a06df66b4ae Jun 02 15:45:52.192191 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.192175 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 15:45:52.228983 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:52.228949 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.255248 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.255205 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" event={"ID":"0b2b7bba8dc600d4bbe83acf7ebb93a3","Type":"ContainerStarted","Data":"20e84dfc0d9407a928a6496880cad57d0ae78fdd0b4c2570ef32ebc71034c127"} Jun 02 15:45:52.256049 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.256029 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" event={"ID":"7e0a2c8ef673f3324a54cfd46ade1a9c","Type":"ContainerStarted","Data":"49476b132dd5abe32a4680724d0dd58ddf93f8a81e6e7eb7a19f0a06df66b4ae"} Jun 02 15:45:52.314984 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.314966 2581 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:45:52.329162 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:52.329140 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.429757 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:52.429656 2581 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-129.ec2.internal\" not found" Jun 02 15:45:52.476916 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.476885 2581 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:45:52.530581 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.530549 2581 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" Jun 02 15:45:52.546169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.546145 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 15:45:52.547567 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.547555 2581 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" Jun 02 15:45:52.561035 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.561010 2581 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 15:45:52.658078 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:52.658039 2581 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:45:53.110398 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.110323 2581 apiserver.go:52] "Watching apiserver" Jun 02 15:45:53.117208 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.117183 2581 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 15:45:53.118421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.118392 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal","openshift-image-registry/node-ca-x8t48","openshift-multus/multus-additional-cni-plugins-w896z","openshift-multus/multus-h56zc","openshift-multus/network-metrics-daemon-rjbmw","openshift-network-operator/iptables-alerter-gwlxx","kube-system/global-pull-secret-syncer-tfkkt","kube-system/konnectivity-agent-tj5r4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m","openshift-cluster-node-tuning-operator/tuned-krpcl","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal","openshift-network-diagnostics/network-check-target-8992p","openshift-ovn-kubernetes/ovnkube-node-vfcmx"] Jun 02 15:45:53.120762 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.120740 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.123474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.123454 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.123608 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.123586 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.123608 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.123599 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-n2wt8\"" Jun 02 15:45:53.123839 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.123821 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.124115 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.124095 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.125044 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.125023 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.126389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.126368 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.126569 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.126542 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.126760 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.126738 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:45:53.127182 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.127162 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.127837 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.127820 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.127937 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.127845 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 15:45:53.127937 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.127890 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 15:45:53.128042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.127989 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.128094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.128042 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-f9t9j\"" Jun 02 15:45:53.128160 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.128143 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-k68x5\"" Jun 02 15:45:53.128379 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.128362 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.128932 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-nqvfx\"" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.128983 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.129004 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.129077 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.129437 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.130437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.129997 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.130978 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.130497 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.130978 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.130505 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-hqfzf\"" Jun 02 15:45:53.131696 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.131677 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:53.131788 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.131737 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:45:53.132111 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132093 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 15:45:53.132190 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132129 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.132190 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132168 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 15:45:53.132413 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132395 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-xntcn\"" Jun 02 15:45:53.132500 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132486 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 15:45:53.132822 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132805 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.132956 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.132941 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.133226 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.133207 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-rdwgn\"" Jun 02 15:45:53.133347 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.133281 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 15:45:53.133410 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.133374 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.134577 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.134562 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.134672 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.134642 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:45:53.135026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.135006 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 15:45:53.135026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.135021 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 15:45:53.135161 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.135036 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-cwc8f\"" Jun 02 15:45:53.135215 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.135157 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 15:45:53.135303 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.135284 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 15:45:53.136173 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.136156 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 15:45:53.136242 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.136162 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 15:45:53.141202 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141179 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.141334 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141215 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2a2d753-3137-48b9-8f03-8781e4ccd858-host-slash\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.141334 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ebf604f8-64aa-471f-b305-bcec9fc03d7c-serviceca\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.141334 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141280 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-cnibin\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.141334 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-system-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141354 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141382 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-socket-dir-parent\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141404 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-k8s-cni-cncf-io\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141423 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-daemon-config\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141448 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-multus-certs\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141465 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-agent-certs\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141490 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141516 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-os-release\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141533 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ebf604f8-64aa-471f-b305-bcec9fc03d7c-host\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.141552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141554 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-run\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bxbfz\" (UniqueName: \"kubernetes.io/projected/db831757-a528-4bc4-ac3e-afebb8ae7fb7-kube-api-access-bxbfz\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141627 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cnibin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141656 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-multus\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-hostroot\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141703 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-conf-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141736 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ndw4p\" (UniqueName: \"kubernetes.io/projected/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-kube-api-access-ndw4p\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141773 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-modprobe-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141804 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-kubernetes\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141842 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-sys\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141869 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-var-lib-kubelet\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141902 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-netns\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141951 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-kubelet\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141974 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kjr94\" (UniqueName: \"kubernetes.io/projected/e2a2d753-3137-48b9-8f03-8781e4ccd858-kube-api-access-kjr94\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.142010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.141997 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-tuned\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142031 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-os-release\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142071 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-binary-copy\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142169 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-95vxl\" (UniqueName: \"kubernetes.io/projected/8f9d8e73-83df-4549-b1a1-5c23188d866d-kube-api-access-95vxl\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142218 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8bsgd\" (UniqueName: \"kubernetes.io/projected/ebf604f8-64aa-471f-b305-bcec9fc03d7c-kube-api-access-8bsgd\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142249 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-lib-modules\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142289 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-host\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142319 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-system-cni-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142357 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysconfig\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-tmp\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142415 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6b44f\" (UniqueName: \"kubernetes.io/projected/ea2e3887-3d5e-4848-9389-595335ede747-kube-api-access-6b44f\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cni-binary-copy\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142469 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-bin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142506 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-etc-kubernetes\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142537 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2a2d753-3137-48b9-8f03-8781e4ccd858-iptables-alerter-script\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142563 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-konnectivity-ca\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.142675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142598 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.143326 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142618 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-conf\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.143326 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142639 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-systemd\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.143326 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.142654 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.178112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.178077 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:40:52 +0000 UTC" deadline="2027-12-29 22:53:04.305808925 +0000 UTC" Jun 02 15:45:53.178220 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.178112 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13807h7m11.127700676s" Jun 02 15:45:53.230709 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.230684 2581 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 15:45:53.243702 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243669 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.243702 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243701 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-conf\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243721 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ebf604f8-64aa-471f-b305-bcec9fc03d7c-serviceca\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243760 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-slash\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243784 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-var-lib-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243806 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-system-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243826 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243853 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-k8s-cni-cncf-io\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243858 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-conf\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243876 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-agent-certs\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243826 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysctl-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243878 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243897 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ebf604f8-64aa-471f-b305-bcec9fc03d7c-host\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.243907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243903 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-k8s-cni-cncf-io\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-system-cni-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243943 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/ebf604f8-64aa-471f-b305-bcec9fc03d7c-host\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243926 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.243982 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244012 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-env-overrides\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244045 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244109 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244146 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mmn7t\" (UniqueName: \"kubernetes.io/projected/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kube-api-access-mmn7t\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244176 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da385797-83ef-450c-93ae-fbc394522e2d-ovn-node-metrics-cert\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244205 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kjr94\" (UniqueName: \"kubernetes.io/projected/e2a2d753-3137-48b9-8f03-8781e4ccd858-kube-api-access-kjr94\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244216 2581 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244231 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-socket-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244254 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-run\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244293 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/ebf604f8-64aa-471f-b305-bcec9fc03d7c-serviceca\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244345 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cnibin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244301 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cnibin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.244501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244392 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-run\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244412 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-multus\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244443 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-conf-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244447 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-multus\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244468 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ndw4p\" (UniqueName: \"kubernetes.io/projected/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-kube-api-access-ndw4p\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244493 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-modprobe-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-conf-dir\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244522 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-kubelet\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244577 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-kubelet\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244594 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-modprobe-d\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244624 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244628 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8bsgd\" (UniqueName: \"kubernetes.io/projected/ebf604f8-64aa-471f-b305-bcec9fc03d7c-kube-api-access-8bsgd\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244680 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-sys-fs\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244707 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9r99d\" (UniqueName: \"kubernetes.io/projected/da385797-83ef-450c-93ae-fbc394522e2d-kube-api-access-9r99d\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244732 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-dbus\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244761 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-binary-copy\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244788 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-node-log\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.245377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244812 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-host\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244839 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-system-cni-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-host\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244863 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-ovn\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244906 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-log-socket\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244921 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-system-cni-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244931 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244951 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-tmp\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244974 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6b44f\" (UniqueName: \"kubernetes.io/projected/ea2e3887-3d5e-4848-9389-595335ede747-kube-api-access-6b44f\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.244993 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-etc-kubernetes\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245020 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2a2d753-3137-48b9-8f03-8781e4ccd858-iptables-alerter-script\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245042 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-konnectivity-ca\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245068 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-netd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245071 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-etc-kubernetes\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245103 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-systemd\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245131 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245158 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.246354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245182 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2a2d753-3137-48b9-8f03-8781e4ccd858-host-slash\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245207 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-systemd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245233 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-cnibin\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245276 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-binary-copy\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245288 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-socket-dir-parent\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245328 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-daemon-config\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245338 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-socket-dir-parent\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245357 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-multus-certs\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245384 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-netns\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245412 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-etc-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245439 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245463 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-config\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245490 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-os-release\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245517 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bxbfz\" (UniqueName: \"kubernetes.io/projected/db831757-a528-4bc4-ac3e-afebb8ae7fb7-kube-api-access-bxbfz\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245542 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-hostroot\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245571 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245584 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-systemd\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247094 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245596 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-bin\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245620 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/e2a2d753-3137-48b9-8f03-8781e4ccd858-iptables-alerter-script\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245624 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-kubernetes\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245667 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-kubernetes\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245666 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-sys\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245713 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-var-lib-kubelet\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245734 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-konnectivity-ca\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245745 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245770 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-netns\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245810 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-tuned\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245811 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-var-lib-kubelet\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245839 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-os-release\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245865 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-sys\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245868 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-95vxl\" (UniqueName: \"kubernetes.io/projected/8f9d8e73-83df-4549-b1a1-5c23188d866d-kube-api-access-95vxl\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245906 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-lib-modules\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245933 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-systemd-units\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245962 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysconfig\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.247806 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.245987 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cni-binary-copy\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246012 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-bin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246067 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-kubelet\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246085 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/ea2e3887-3d5e-4848-9389-595335ede747-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246140 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-sysconfig\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246171 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-tuning-conf-dir\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.246183 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246183 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-multus-daemon-config\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246090 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-script-lib\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246224 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-cnibin\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246224 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-kubelet-config\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-var-lib-cni-bin\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246305 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/e2a2d753-3137-48b9-8f03-8781e4ccd858-host-slash\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246312 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-os-release\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.246373 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:45:53.746321539 +0000 UTC m=+3.020753598 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246417 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/db831757-a528-4bc4-ac3e-afebb8ae7fb7-lib-modules\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.248374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246451 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-multus-certs\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246516 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-host-run-netns\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246529 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/ea2e3887-3d5e-4848-9389-595335ede747-os-release\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246563 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-hostroot\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-registration-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246597 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-device-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.246687 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-cni-binary-copy\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.248137 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/c11ecb1c-64f7-42bc-bcd7-f09b91558d0c-agent-certs\") pod \"konnectivity-agent-tj5r4\" (UID: \"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c\") " pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.248386 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-tmp\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.248912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.248757 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/db831757-a528-4bc4-ac3e-afebb8ae7fb7-etc-tuned\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.255241 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.255204 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ndw4p\" (UniqueName: \"kubernetes.io/projected/3e3faefc-0181-4a03-8b2c-6038fe0cb8a7-kube-api-access-ndw4p\") pod \"multus-h56zc\" (UID: \"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7\") " pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.255801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.255561 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8bsgd\" (UniqueName: \"kubernetes.io/projected/ebf604f8-64aa-471f-b305-bcec9fc03d7c-kube-api-access-8bsgd\") pod \"node-ca-x8t48\" (UID: \"ebf604f8-64aa-471f-b305-bcec9fc03d7c\") " pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.255999 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.255982 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kjr94\" (UniqueName: \"kubernetes.io/projected/e2a2d753-3137-48b9-8f03-8781e4ccd858-kube-api-access-kjr94\") pod \"iptables-alerter-gwlxx\" (UID: \"e2a2d753-3137-48b9-8f03-8781e4ccd858\") " pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.257101 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.257083 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bxbfz\" (UniqueName: \"kubernetes.io/projected/db831757-a528-4bc4-ac3e-afebb8ae7fb7-kube-api-access-bxbfz\") pod \"tuned-krpcl\" (UID: \"db831757-a528-4bc4-ac3e-afebb8ae7fb7\") " pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.257893 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.257873 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-95vxl\" (UniqueName: \"kubernetes.io/projected/8f9d8e73-83df-4549-b1a1-5c23188d866d-kube-api-access-95vxl\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.259011 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.258994 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6b44f\" (UniqueName: \"kubernetes.io/projected/ea2e3887-3d5e-4848-9389-595335ede747-kube-api-access-6b44f\") pod \"multus-additional-cni-plugins-w896z\" (UID: \"ea2e3887-3d5e-4848-9389-595335ede747\") " pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.346907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.346870 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-netd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.346922 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-systemd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.346946 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-netns\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.346991 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-netns\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347002 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-netd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.346992 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-systemd\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347037 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-etc-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347011 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-etc-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347065 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347089 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-config\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347111 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347118 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347140 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-bin\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347159 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-systemd-units\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347177 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347192 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-kubelet\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347202 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-run-ovn-kubernetes\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347206 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-script-lib\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347225 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-cni-bin\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347234 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-kubelet\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347252 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-systemd-units\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347255 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-kubelet-config\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347383 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-registration-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347398 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-kubelet-config\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347434 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-device-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347477 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-registration-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347505 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-device-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347513 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-slash\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347546 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-var-lib-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347578 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347603 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347627 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-env-overrides\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347638 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347652 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347676 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mmn7t\" (UniqueName: \"kubernetes.io/projected/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kube-api-access-mmn7t\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347604 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-var-lib-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347695 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-config\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347705 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da385797-83ef-450c-93ae-fbc394522e2d-ovn-node-metrics-cert\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.347979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347741 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-socket-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347774 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-sys-fs\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347778 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-etc-selinux\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347799 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9r99d\" (UniqueName: \"kubernetes.io/projected/da385797-83ef-450c-93ae-fbc394522e2d-kube-api-access-9r99d\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347824 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-dbus\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347849 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-node-log\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347854 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-ovnkube-script-lib\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347567 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-host-slash\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347874 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-ovn\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347939 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-log-socket\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347963 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347997 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-socket-dir\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.348048 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348058 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f21bbe84-c4b9-456a-b20a-8172a468bd7b-sys-fs\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.347678 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-openvswitch\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.348099 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:53.848082713 +0000 UTC m=+3.122514757 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348099 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-node-log\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.348746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348155 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-log-socket\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.349557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348200 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/da385797-83ef-450c-93ae-fbc394522e2d-run-ovn\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.349557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348295 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-dbus\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.349557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.348480 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/da385797-83ef-450c-93ae-fbc394522e2d-env-overrides\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.350548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.350528 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/da385797-83ef-450c-93ae-fbc394522e2d-ovn-node-metrics-cert\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.354441 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.354420 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:45:53.354546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.354447 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:45:53.354546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.354458 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:53.354546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.354536 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:53.854519163 +0000 UTC m=+3.128951250 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:53.360365 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.360346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mmn7t\" (UniqueName: \"kubernetes.io/projected/f21bbe84-c4b9-456a-b20a-8172a468bd7b-kube-api-access-mmn7t\") pod \"aws-ebs-csi-driver-node-tzw5m\" (UID: \"f21bbe84-c4b9-456a-b20a-8172a468bd7b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.360535 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.360495 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9r99d\" (UniqueName: \"kubernetes.io/projected/da385797-83ef-450c-93ae-fbc394522e2d-kube-api-access-9r99d\") pod \"ovnkube-node-vfcmx\" (UID: \"da385797-83ef-450c-93ae-fbc394522e2d\") " pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.434980 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.434944 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-krpcl" Jun 02 15:45:53.440805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.440781 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-x8t48" Jun 02 15:45:53.449359 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.449325 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-w896z" Jun 02 15:45:53.455010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.454988 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-h56zc" Jun 02 15:45:53.463563 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.463537 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-gwlxx" Jun 02 15:45:53.470122 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.470101 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:45:53.479685 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.479658 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" Jun 02 15:45:53.484282 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.484251 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:45:53.592281 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.592232 2581 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 15:45:53.750657 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.750627 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:53.750769 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.750734 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:53.750851 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.750788 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:45:54.750771574 +0000 UTC m=+4.025203631 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:53.770662 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.770632 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podea2e3887_3d5e_4848_9389_595335ede747.slice/crio-2e37f5be2e96186b5c2f41640d441e5f81059d11aa016cb3eec1e1567a3d11fa WatchSource:0}: Error finding container 2e37f5be2e96186b5c2f41640d441e5f81059d11aa016cb3eec1e1567a3d11fa: Status 404 returned error can't find the container with id 2e37f5be2e96186b5c2f41640d441e5f81059d11aa016cb3eec1e1567a3d11fa Jun 02 15:45:53.772174 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.772149 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podda385797_83ef_450c_93ae_fbc394522e2d.slice/crio-38e087626635d4d9fc9f3b4c817d56574dce80ae1c4557d6fa56c00c9ffd8682 WatchSource:0}: Error finding container 38e087626635d4d9fc9f3b4c817d56574dce80ae1c4557d6fa56c00c9ffd8682: Status 404 returned error can't find the container with id 38e087626635d4d9fc9f3b4c817d56574dce80ae1c4557d6fa56c00c9ffd8682 Jun 02 15:45:53.773019 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.772996 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf21bbe84_c4b9_456a_b20a_8172a468bd7b.slice/crio-ec1b7513ce2e6f0f9992b3d1d474726ff183ef32e56aa3dbbc74d5d39f951177 WatchSource:0}: Error finding container ec1b7513ce2e6f0f9992b3d1d474726ff183ef32e56aa3dbbc74d5d39f951177: Status 404 returned error can't find the container with id ec1b7513ce2e6f0f9992b3d1d474726ff183ef32e56aa3dbbc74d5d39f951177 Jun 02 15:45:53.774687 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.774640 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode2a2d753_3137_48b9_8f03_8781e4ccd858.slice/crio-7719303ffb8d5a45c6d60d3c2f2680c4dd301dc8a5cac2ff8ee2177b031267a5 WatchSource:0}: Error finding container 7719303ffb8d5a45c6d60d3c2f2680c4dd301dc8a5cac2ff8ee2177b031267a5: Status 404 returned error can't find the container with id 7719303ffb8d5a45c6d60d3c2f2680c4dd301dc8a5cac2ff8ee2177b031267a5 Jun 02 15:45:53.777599 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.777557 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e3faefc_0181_4a03_8b2c_6038fe0cb8a7.slice/crio-a3e0e9b993ab25967e1ff99d01cd311675217d20caf572a3aad010ebd4f6bd94 WatchSource:0}: Error finding container a3e0e9b993ab25967e1ff99d01cd311675217d20caf572a3aad010ebd4f6bd94: Status 404 returned error can't find the container with id a3e0e9b993ab25967e1ff99d01cd311675217d20caf572a3aad010ebd4f6bd94 Jun 02 15:45:53.778105 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.778055 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc11ecb1c_64f7_42bc_bcd7_f09b91558d0c.slice/crio-6957a634d0b8459b2b3e442c58a98125f51eddc626e862be9eb2b5ed0ab3deec WatchSource:0}: Error finding container 6957a634d0b8459b2b3e442c58a98125f51eddc626e862be9eb2b5ed0ab3deec: Status 404 returned error can't find the container with id 6957a634d0b8459b2b3e442c58a98125f51eddc626e862be9eb2b5ed0ab3deec Jun 02 15:45:53.779203 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.779071 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddb831757_a528_4bc4_ac3e_afebb8ae7fb7.slice/crio-d6f2da13d0928cfcabbb2feadadd6f813259fefd3b5f384afb4d890a8a35fa9d WatchSource:0}: Error finding container d6f2da13d0928cfcabbb2feadadd6f813259fefd3b5f384afb4d890a8a35fa9d: Status 404 returned error can't find the container with id d6f2da13d0928cfcabbb2feadadd6f813259fefd3b5f384afb4d890a8a35fa9d Jun 02 15:45:53.780699 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:45:53.780635 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebf604f8_64aa_471f_b305_bcec9fc03d7c.slice/crio-d0e1ca24455e19f340033bbd2220fb2fd30c9f75ea2c4ea46f7d0673fca75e9c WatchSource:0}: Error finding container d0e1ca24455e19f340033bbd2220fb2fd30c9f75ea2c4ea46f7d0673fca75e9c: Status 404 returned error can't find the container with id d0e1ca24455e19f340033bbd2220fb2fd30c9f75ea2c4ea46f7d0673fca75e9c Jun 02 15:45:53.851400 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.851232 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:53.851526 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.851356 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:53.851526 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.851508 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:54.851489059 +0000 UTC m=+4.125921113 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:53.952377 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:53.952346 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:53.952515 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.952475 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:45:53.952515 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.952494 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:45:53.952515 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.952504 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:53.952661 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:53.952556 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:54.952540693 +0000 UTC m=+4.226972737 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:54.178338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.178253 2581 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 15:40:52 +0000 UTC" deadline="2028-01-03 02:23:32.033391477 +0000 UTC" Jun 02 15:45:54.178338 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.178303 2581 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13906h37m37.85509218s" Jun 02 15:45:54.252762 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.252687 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:54.252920 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.252819 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:45:54.264584 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.264548 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-krpcl" event={"ID":"db831757-a528-4bc4-ac3e-afebb8ae7fb7","Type":"ContainerStarted","Data":"d6f2da13d0928cfcabbb2feadadd6f813259fefd3b5f384afb4d890a8a35fa9d"} Jun 02 15:45:54.268889 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.268821 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h56zc" event={"ID":"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7","Type":"ContainerStarted","Data":"a3e0e9b993ab25967e1ff99d01cd311675217d20caf572a3aad010ebd4f6bd94"} Jun 02 15:45:54.281674 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.281618 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwlxx" event={"ID":"e2a2d753-3137-48b9-8f03-8781e4ccd858","Type":"ContainerStarted","Data":"7719303ffb8d5a45c6d60d3c2f2680c4dd301dc8a5cac2ff8ee2177b031267a5"} Jun 02 15:45:54.285500 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.285444 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" event={"ID":"f21bbe84-c4b9-456a-b20a-8172a468bd7b","Type":"ContainerStarted","Data":"ec1b7513ce2e6f0f9992b3d1d474726ff183ef32e56aa3dbbc74d5d39f951177"} Jun 02 15:45:54.293113 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.293083 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"38e087626635d4d9fc9f3b4c817d56574dce80ae1c4557d6fa56c00c9ffd8682"} Jun 02 15:45:54.295728 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.295613 2581 generic.go:358] "Generic (PLEG): container finished" podID="0b2b7bba8dc600d4bbe83acf7ebb93a3" containerID="9f54f6fccfa6507298d47042cc6ace3c0846757d1d882343e040f7b993d07688" exitCode=0 Jun 02 15:45:54.295728 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.295690 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" event={"ID":"0b2b7bba8dc600d4bbe83acf7ebb93a3","Type":"ContainerDied","Data":"9f54f6fccfa6507298d47042cc6ace3c0846757d1d882343e040f7b993d07688"} Jun 02 15:45:54.303865 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.303835 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" event={"ID":"7e0a2c8ef673f3324a54cfd46ade1a9c","Type":"ContainerStarted","Data":"b6c72d743596820a18d292203a7383797de078c9fbe6044e4df9b12e72b3bfd1"} Jun 02 15:45:54.308089 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.308063 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tj5r4" event={"ID":"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c","Type":"ContainerStarted","Data":"6957a634d0b8459b2b3e442c58a98125f51eddc626e862be9eb2b5ed0ab3deec"} Jun 02 15:45:54.310243 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.310217 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8t48" event={"ID":"ebf604f8-64aa-471f-b305-bcec9fc03d7c","Type":"ContainerStarted","Data":"d0e1ca24455e19f340033bbd2220fb2fd30c9f75ea2c4ea46f7d0673fca75e9c"} Jun 02 15:45:54.318536 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.318502 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerStarted","Data":"2e37f5be2e96186b5c2f41640d441e5f81059d11aa016cb3eec1e1567a3d11fa"} Jun 02 15:45:54.336005 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.335946 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-129.ec2.internal" podStartSLOduration=2.33592904 podStartE2EDuration="2.33592904s" podCreationTimestamp="2026-06-02 15:45:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:45:54.334937742 +0000 UTC m=+3.609369809" watchObservedRunningTime="2026-06-02 15:45:54.33592904 +0000 UTC m=+3.610361106" Jun 02 15:45:54.759257 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.759171 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:54.759430 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.759365 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:54.759507 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.759440 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:45:56.759411415 +0000 UTC m=+6.033843466 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:54.860236 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.860198 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:54.860433 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.860369 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:54.860490 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.860434 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:56.86041643 +0000 UTC m=+6.134848478 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:54.961523 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:54.961481 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:54.961715 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.961650 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:45:54.961715 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.961672 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:45:54.961715 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.961685 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:54.961866 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:54.961741 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:45:56.961724005 +0000 UTC m=+6.236156056 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:55.255368 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:55.255334 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:55.255826 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:55.255474 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:45:55.255919 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:55.255901 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:55.256019 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:55.255997 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:45:55.342070 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:55.341962 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" event={"ID":"0b2b7bba8dc600d4bbe83acf7ebb93a3","Type":"ContainerStarted","Data":"159723154d8eb29cf1b312d30f45684709c34530dbf4fe3c383637039b5a5ace"} Jun 02 15:45:55.356531 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:55.356448 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-129.ec2.internal" podStartSLOduration=3.356431231 podStartE2EDuration="3.356431231s" podCreationTimestamp="2026-06-02 15:45:52 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:45:55.35641358 +0000 UTC m=+4.630845645" watchObservedRunningTime="2026-06-02 15:45:55.356431231 +0000 UTC m=+4.630863297" Jun 02 15:45:56.253523 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:56.252996 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:56.253523 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.253126 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:45:56.778280 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:56.778224 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:56.778684 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.778404 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:56.778684 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.778486 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:46:00.778466763 +0000 UTC m=+10.052898806 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:45:56.879594 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:56.879558 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:56.879765 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.879748 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:56.879828 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.879815 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:00.879795276 +0000 UTC m=+10.154227326 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:45:56.980169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:56.980129 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:56.980361 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.980304 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:45:56.980361 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.980324 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:45:56.980361 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.980336 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:56.980524 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:56.980395 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:00.980375444 +0000 UTC m=+10.254807487 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:45:57.252456 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:57.252377 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:57.252456 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:57.252399 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:57.252705 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:57.252492 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:45:57.252705 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:57.252683 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:45:58.252775 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:58.252208 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:45:58.252775 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:58.252369 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:45:59.255107 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:59.255077 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:45:59.255524 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:59.255208 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:45:59.255620 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:45:59.255604 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:45:59.255709 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:45:59.255690 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:00.252432 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:00.252400 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:00.252582 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:00.252515 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:00.810801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:00.810762 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:00.811307 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:00.810900 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:00.811307 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:00.810972 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:46:08.810951856 +0000 UTC m=+18.085383928 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:00.911636 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:00.911592 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:00.911806 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:00.911746 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:00.911859 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:00.911822 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:08.911803016 +0000 UTC m=+18.186235060 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:01.013317 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:01.012792 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:01.013317 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.012954 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:01.013317 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.012971 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:01.013317 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.012980 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:01.013317 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.013026 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:09.013013339 +0000 UTC m=+18.287445382 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:01.253498 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:01.253461 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:01.253665 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.253575 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:01.253745 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:01.253667 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:01.253813 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:01.253788 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:02.252746 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:02.252697 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:02.253223 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:02.252854 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:03.252495 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:03.252458 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:03.252680 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:03.252458 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:03.252680 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:03.252591 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:03.252680 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:03.252635 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:04.252024 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:04.251986 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:04.252202 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:04.252124 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:05.252246 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:05.252205 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:05.252640 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:05.252351 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:05.252640 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:05.252219 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:05.252640 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:05.252438 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:06.252424 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:06.252393 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:06.252834 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:06.252507 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:07.252471 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:07.252435 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:07.253001 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:07.252434 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:07.253001 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:07.252551 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:07.253001 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:07.252685 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:08.252529 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:08.252494 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:08.252861 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:08.252633 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:08.869425 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:08.869390 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:08.869614 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:08.869520 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:08.869614 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:08.869579 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:46:24.869565216 +0000 UTC m=+34.143997259 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:08.970008 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:08.969959 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:08.970164 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:08.970136 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:08.970227 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:08.970216 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:24.970195032 +0000 UTC m=+34.244627089 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:09.070537 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:09.070495 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:09.070723 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.070639 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:09.070723 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.070660 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:09.070723 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.070674 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:09.070850 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.070734 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:25.070715549 +0000 UTC m=+34.345147602 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:09.252722 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:09.252679 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:09.253147 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:09.252694 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:09.253147 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.252841 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:09.253147 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:09.252886 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:10.252402 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:10.252365 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:10.252591 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:10.252505 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:11.253611 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:11.253571 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:11.253611 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:11.253607 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:11.254123 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:11.253686 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:11.254123 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:11.253782 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:12.252717 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.252548 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:12.252814 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:12.252796 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:12.370991 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.370825 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="ece56fd76e637d47c103ff9a9e48807e0748815e44a8437bedff3e20abf54147" exitCode=0 Jun 02 15:46:12.371740 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.370909 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"ece56fd76e637d47c103ff9a9e48807e0748815e44a8437bedff3e20abf54147"} Jun 02 15:46:12.372550 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.372522 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-krpcl" event={"ID":"db831757-a528-4bc4-ac3e-afebb8ae7fb7","Type":"ContainerStarted","Data":"0831c584b11e6e5bb423dc8f191dd3c46d6eedf15f2da35d0d723356c9b9a0e0"} Jun 02 15:46:12.373761 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.373736 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-h56zc" event={"ID":"3e3faefc-0181-4a03-8b2c-6038fe0cb8a7","Type":"ContainerStarted","Data":"d46df811f7afc99030cde18b3aa80a1be07a336a206381a384dd52cb5b6c9504"} Jun 02 15:46:12.375068 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.375048 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" event={"ID":"f21bbe84-c4b9-456a-b20a-8172a468bd7b","Type":"ContainerStarted","Data":"8002bac94718bf8b6fa457974aa4dd01d6cc570299b93ecde0294e873e9c1572"} Jun 02 15:46:12.376727 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.376711 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:46:12.377074 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.377049 2581 generic.go:358] "Generic (PLEG): container finished" podID="da385797-83ef-450c-93ae-fbc394522e2d" containerID="097f1111dd7e5c8b85941373faafffcb287995fc0d0f74708964b9dee4bdef05" exitCode=1 Jun 02 15:46:12.377153 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.377130 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"efead734e4ed38aa84d4cd06dbf8b6a3cb92dc5a07fc238ebd59e19050fbbc6e"} Jun 02 15:46:12.377234 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.377162 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerDied","Data":"097f1111dd7e5c8b85941373faafffcb287995fc0d0f74708964b9dee4bdef05"} Jun 02 15:46:12.377234 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.377178 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"31496c6fdf15f3e4630847ce60d33236c04b1cdc8d956c75c0ee6ba53ab76541"} Jun 02 15:46:12.378385 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.378363 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-tj5r4" event={"ID":"c11ecb1c-64f7-42bc-bcd7-f09b91558d0c","Type":"ContainerStarted","Data":"d4e5e97ebeedbc84d087ca8dea94971381a1d80229db24ed6d47ed6819530357"} Jun 02 15:46:12.380661 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.380639 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-x8t48" event={"ID":"ebf604f8-64aa-471f-b305-bcec9fc03d7c","Type":"ContainerStarted","Data":"e69d07317c7f077ad6b20c53c0e46375cefd341ca9ef33f8bc2a5b4ac2b6e3b8"} Jun 02 15:46:12.416501 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.416450 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-krpcl" podStartSLOduration=3.4748533520000002 podStartE2EDuration="21.416437308s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.78274518 +0000 UTC m=+3.057177231" lastFinishedPulling="2026-06-02 15:46:11.724329143 +0000 UTC m=+20.998761187" observedRunningTime="2026-06-02 15:46:12.4159774 +0000 UTC m=+21.690409464" watchObservedRunningTime="2026-06-02 15:46:12.416437308 +0000 UTC m=+21.690869372" Jun 02 15:46:12.432562 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.432522 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-x8t48" podStartSLOduration=3.493375187 podStartE2EDuration="21.432507288s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.783431413 +0000 UTC m=+3.057863471" lastFinishedPulling="2026-06-02 15:46:11.722563528 +0000 UTC m=+20.996995572" observedRunningTime="2026-06-02 15:46:12.431957881 +0000 UTC m=+21.706389959" watchObservedRunningTime="2026-06-02 15:46:12.432507288 +0000 UTC m=+21.706939352" Jun 02 15:46:12.451074 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.451021 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-h56zc" podStartSLOduration=3.255775771 podStartE2EDuration="21.451006457s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.778977699 +0000 UTC m=+3.053409758" lastFinishedPulling="2026-06-02 15:46:11.974208401 +0000 UTC m=+21.248640444" observedRunningTime="2026-06-02 15:46:12.450471308 +0000 UTC m=+21.724903373" watchObservedRunningTime="2026-06-02 15:46:12.451006457 +0000 UTC m=+21.725438521" Jun 02 15:46:12.473134 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:12.473075 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-tj5r4" podStartSLOduration=3.5658835079999998 podStartE2EDuration="21.473064575s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.780698388 +0000 UTC m=+3.055130434" lastFinishedPulling="2026-06-02 15:46:11.687879457 +0000 UTC m=+20.962311501" observedRunningTime="2026-06-02 15:46:12.472738634 +0000 UTC m=+21.747170698" watchObservedRunningTime="2026-06-02 15:46:12.473064575 +0000 UTC m=+21.747496639" Jun 02 15:46:13.112889 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.112844 2581 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 15:46:13.202973 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.202863 2581 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T15:46:13.112869608Z","UUID":"f2daa74e-0221-4b23-91c3-4cacfe53984f","Handler":null,"Name":"","Endpoint":""} Jun 02 15:46:13.204754 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.204731 2581 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 15:46:13.204873 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.204763 2581 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 15:46:13.252936 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.252897 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:13.252936 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.252938 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:13.253182 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:13.253045 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:13.253182 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:13.253174 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:13.384143 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.384048 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" event={"ID":"f21bbe84-c4b9-456a-b20a-8172a468bd7b","Type":"ContainerStarted","Data":"903a94479015b747701270fccd7732f6ceae54a572b452bc795125379b452735"} Jun 02 15:46:13.386985 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.386962 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:46:13.387363 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.387325 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"8b727ae80767476072f3ab5656f757f8868f9dec0946b1c9042e0080806f1eb8"} Jun 02 15:46:13.387468 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.387373 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"7acf811d4d492badf6cbd01332e185d40e5acce21ffa95ed028ac1159ac82c89"} Jun 02 15:46:13.387468 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.387389 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"721edea35dcbb8d825c226b37272538bf3b258d664dfcb30434125993d99eda4"} Jun 02 15:46:13.388782 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.388756 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-gwlxx" event={"ID":"e2a2d753-3137-48b9-8f03-8781e4ccd858","Type":"ContainerStarted","Data":"00ef0642b1a443d24796d20decb48ac25be4d71e6887c5e31d6e48dac04a32cb"} Jun 02 15:46:13.406753 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:13.406706 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-gwlxx" podStartSLOduration=4.460539088 podStartE2EDuration="22.406694087s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.776406798 +0000 UTC m=+3.050838844" lastFinishedPulling="2026-06-02 15:46:11.722561792 +0000 UTC m=+20.996993843" observedRunningTime="2026-06-02 15:46:13.406428776 +0000 UTC m=+22.680860845" watchObservedRunningTime="2026-06-02 15:46:13.406694087 +0000 UTC m=+22.681126152" Jun 02 15:46:14.252023 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:14.251992 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:14.252166 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:14.252114 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:14.392385 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:14.392347 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" event={"ID":"f21bbe84-c4b9-456a-b20a-8172a468bd7b","Type":"ContainerStarted","Data":"ca35ae7391e66f10c149891e69dea8a8ed02e90f0d1232d0ae23b0403934a473"} Jun 02 15:46:15.103953 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.103713 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-tzw5m" podStartSLOduration=3.732327285 podStartE2EDuration="24.103696368s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.776008528 +0000 UTC m=+3.050440576" lastFinishedPulling="2026-06-02 15:46:14.147377613 +0000 UTC m=+23.421809659" observedRunningTime="2026-06-02 15:46:14.411060055 +0000 UTC m=+23.685492121" watchObservedRunningTime="2026-06-02 15:46:15.103696368 +0000 UTC m=+24.378128470" Jun 02 15:46:15.104359 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.104325 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-p7f6j"] Jun 02 15:46:15.115435 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.115403 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.117942 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.117915 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 15:46:15.118071 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.118046 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 15:46:15.118182 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.118161 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-9v94l\"" Jun 02 15:46:15.214319 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.214283 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dad6529-26f1-4882-adc3-f3e371520b17-tmp-dir\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.214512 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.214361 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dad6529-26f1-4882-adc3-f3e371520b17-hosts-file\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.214512 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.214390 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xklxx\" (UniqueName: \"kubernetes.io/projected/2dad6529-26f1-4882-adc3-f3e371520b17-kube-api-access-xklxx\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.252414 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.252383 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:15.252618 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:15.252519 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:15.252618 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.252587 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:15.252743 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:15.252709 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:15.315180 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.315144 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dad6529-26f1-4882-adc3-f3e371520b17-hosts-file\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.315180 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.315182 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xklxx\" (UniqueName: \"kubernetes.io/projected/2dad6529-26f1-4882-adc3-f3e371520b17-kube-api-access-xklxx\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.315422 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.315241 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dad6529-26f1-4882-adc3-f3e371520b17-tmp-dir\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.315422 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.315321 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/2dad6529-26f1-4882-adc3-f3e371520b17-hosts-file\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.315879 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.315856 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/2dad6529-26f1-4882-adc3-f3e371520b17-tmp-dir\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.326197 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.326167 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xklxx\" (UniqueName: \"kubernetes.io/projected/2dad6529-26f1-4882-adc3-f3e371520b17-kube-api-access-xklxx\") pod \"node-resolver-p7f6j\" (UID: \"2dad6529-26f1-4882-adc3-f3e371520b17\") " pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:15.396389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.396313 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:46:15.396867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.396620 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"033e149c878f93ffec25a5de06774527fb301adfeda4007b265c89a2aa3cf0fa"} Jun 02 15:46:15.425286 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:15.425231 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-p7f6j" Jun 02 15:46:16.252611 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:16.252578 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:16.252796 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:16.252698 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:16.465828 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:16.465786 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2dad6529_26f1_4882_adc3_f3e371520b17.slice/crio-1bb352c68c75b6a335dbf9173604483f51caaa6f12459c658fba7bdbc8aa09f4 WatchSource:0}: Error finding container 1bb352c68c75b6a335dbf9173604483f51caaa6f12459c658fba7bdbc8aa09f4: Status 404 returned error can't find the container with id 1bb352c68c75b6a335dbf9173604483f51caaa6f12459c658fba7bdbc8aa09f4 Jun 02 15:46:17.252385 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.252196 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:17.252589 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.252216 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:17.252589 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:17.252468 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:17.252589 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:17.252577 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:17.337844 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.337808 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:46:17.338413 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.338395 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:46:17.402781 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.402755 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:46:17.403070 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.403042 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"0700fe22236cd3b87a94fdcb64e8db4007e52b2a55d4c1bf095b3a80928a2976"} Jun 02 15:46:17.403334 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.403312 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:17.403444 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.403341 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:17.403511 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.403497 2581 scope.go:117] "RemoveContainer" containerID="097f1111dd7e5c8b85941373faafffcb287995fc0d0f74708964b9dee4bdef05" Jun 02 15:46:17.404450 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.404421 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p7f6j" event={"ID":"2dad6529-26f1-4882-adc3-f3e371520b17","Type":"ContainerStarted","Data":"300827a82445f9a1351f92a21dc464ab8bccac7a752a10643d4b0f3a83b11c08"} Jun 02 15:46:17.404526 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.404464 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-p7f6j" event={"ID":"2dad6529-26f1-4882-adc3-f3e371520b17","Type":"ContainerStarted","Data":"1bb352c68c75b6a335dbf9173604483f51caaa6f12459c658fba7bdbc8aa09f4"} Jun 02 15:46:17.406171 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.406151 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="8cf3a8f9da02986912ef91619c703778a7667009a770ae2ea9b82123e4dc0455" exitCode=0 Jun 02 15:46:17.406258 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.406225 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"8cf3a8f9da02986912ef91619c703778a7667009a770ae2ea9b82123e4dc0455"} Jun 02 15:46:17.406545 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.406524 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:46:17.407045 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.406974 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-tj5r4" Jun 02 15:46:17.420795 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.420776 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:17.470997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:17.470955 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-p7f6j" podStartSLOduration=2.470941807 podStartE2EDuration="2.470941807s" podCreationTimestamp="2026-06-02 15:46:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:46:17.448988213 +0000 UTC m=+26.723420277" watchObservedRunningTime="2026-06-02 15:46:17.470941807 +0000 UTC m=+26.745373871" Jun 02 15:46:18.252125 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.251961 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:18.252125 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:18.252085 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:18.411386 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.411360 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:46:18.411739 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.411701 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" event={"ID":"da385797-83ef-450c-93ae-fbc394522e2d","Type":"ContainerStarted","Data":"9962f25fdc936e08e4de5530cce8899638b3ba6b05dce4d3dd3a5ce3ad57bd3d"} Jun 02 15:46:18.412072 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.412051 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:18.413975 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.413951 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="fde8bda7443eb9f9609f799bb0717b6953267400a615b826ea77b7674b070a79" exitCode=0 Jun 02 15:46:18.414071 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.414037 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"fde8bda7443eb9f9609f799bb0717b6953267400a615b826ea77b7674b070a79"} Jun 02 15:46:18.428748 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.428725 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:18.441197 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.441151 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" podStartSLOduration=9.444925726 podStartE2EDuration="27.44113841s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.773965208 +0000 UTC m=+3.048397265" lastFinishedPulling="2026-06-02 15:46:11.770177905 +0000 UTC m=+21.044609949" observedRunningTime="2026-06-02 15:46:18.440068873 +0000 UTC m=+27.714500940" watchObservedRunningTime="2026-06-02 15:46:18.44113841 +0000 UTC m=+27.715570506" Jun 02 15:46:18.771048 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.771009 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfkkt"] Jun 02 15:46:18.771514 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.771118 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:18.771514 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:18.771214 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:18.773249 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.773225 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8992p"] Jun 02 15:46:18.773371 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.773351 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:18.773463 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:18.773444 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:18.784458 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.784425 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjbmw"] Jun 02 15:46:18.784599 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:18.784559 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:18.784704 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:18.784671 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:19.418520 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:19.418478 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="83224129739f703412d0c7448adc39c294f9e8ed366b80743a66a14f4aaf6c2b" exitCode=0 Jun 02 15:46:19.418684 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:19.418560 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"83224129739f703412d0c7448adc39c294f9e8ed366b80743a66a14f4aaf6c2b"} Jun 02 15:46:20.252802 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:20.252763 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:20.253634 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:20.252822 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:20.253634 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:20.252920 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:20.253634 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:20.253013 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:20.253634 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:20.253077 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:20.253634 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:20.253188 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:22.252823 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:22.252784 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:22.252823 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:22.252818 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:22.253563 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:22.252788 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:22.253563 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:22.252928 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:22.253563 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:22.252995 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:22.253563 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:22.253041 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:24.252177 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.252135 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:24.252852 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.252153 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:24.252852 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.252293 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-tfkkt" podUID="ff60fcd6-3fa3-4be1-b821-fdebcadc7675" Jun 02 15:46:24.252852 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.252310 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:24.252852 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.252400 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-8992p" podUID="a548ced1-3bed-4d35-a80d-8aa03e106398" Jun 02 15:46:24.252852 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.252494 2581 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-rjbmw" podUID="8f9d8e73-83df-4549-b1a1-5c23188d866d" Jun 02 15:46:24.525974 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.525897 2581 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-129.ec2.internal" event="NodeReady" Jun 02 15:46:24.526138 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.526041 2581 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 15:46:24.585831 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.585792 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-fs7v5"] Jun 02 15:46:24.612375 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.612343 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-5kpc8"] Jun 02 15:46:24.630087 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.630028 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fs7v5"] Jun 02 15:46:24.630087 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.630075 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5kpc8"] Jun 02 15:46:24.630343 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.630120 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.630424 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.630405 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.638782 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.638523 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 15:46:24.638985 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.638597 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 15:46:24.638985 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.638719 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpsk2\"" Jun 02 15:46:24.639493 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.639290 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 15:46:24.639835 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.639814 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 15:46:24.639938 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.639890 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s8ml8\"" Jun 02 15:46:24.640257 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.640231 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 15:46:24.689279 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.689226 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.689443 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.689336 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gl4td\" (UniqueName: \"kubernetes.io/projected/94c86a42-2dd5-4552-8585-01550f4f9736-kube-api-access-gl4td\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.790662 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790541 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-tmp-dir\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.790662 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790642 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gl4td\" (UniqueName: \"kubernetes.io/projected/94c86a42-2dd5-4552-8585-01550f4f9736-kube-api-access-gl4td\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.790872 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790674 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-config-volume\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.790872 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790730 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.790872 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790754 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.790872 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.790775 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qn5r8\" (UniqueName: \"kubernetes.io/projected/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-kube-api-access-qn5r8\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.791006 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.790906 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:24.791006 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.790974 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:25.290958329 +0000 UTC m=+34.565390383 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:24.802570 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.802540 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gl4td\" (UniqueName: \"kubernetes.io/projected/94c86a42-2dd5-4552-8585-01550f4f9736-kube-api-access-gl4td\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:24.891527 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.891485 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-config-volume\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.891723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.891583 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.891723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.891611 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qn5r8\" (UniqueName: \"kubernetes.io/projected/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-kube-api-access-qn5r8\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.891723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.891644 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-tmp-dir\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.891723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.891688 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:24.891891 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.891808 2581 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:24.891891 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.891870 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs podName:8f9d8e73-83df-4549-b1a1-5c23188d866d nodeName:}" failed. No retries permitted until 2026-06-02 15:46:56.891849586 +0000 UTC m=+66.166281644 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs") pod "network-metrics-daemon-rjbmw" (UID: "8f9d8e73-83df-4549-b1a1-5c23188d866d") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 15:46:24.892170 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.892141 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:24.892170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.892152 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-config-volume\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.892363 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.892200 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:25.392182281 +0000 UTC m=+34.666614328 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:24.892363 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.892324 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-tmp-dir\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.912717 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.912682 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qn5r8\" (UniqueName: \"kubernetes.io/projected/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-kube-api-access-qn5r8\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:24.992330 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:24.992290 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:24.992487 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.992403 2581 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:24.992487 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:24.992453 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret podName:ff60fcd6-3fa3-4be1-b821-fdebcadc7675 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:56.992441546 +0000 UTC m=+66.266873592 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret") pod "global-pull-secret-syncer-tfkkt" (UID: "ff60fcd6-3fa3-4be1-b821-fdebcadc7675") : object "kube-system"/"original-pull-secret" not registered Jun 02 15:46:25.093375 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:25.093285 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:25.093546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.093382 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 15:46:25.093546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.093405 2581 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 15:46:25.093546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.093415 2581 projected.go:194] Error preparing data for projected volume kube-api-access-lkhfh for pod openshift-network-diagnostics/network-check-target-8992p: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:25.093546 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.093470 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh podName:a548ced1-3bed-4d35-a80d-8aa03e106398 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:57.093456514 +0000 UTC m=+66.367888561 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-lkhfh" (UniqueName: "kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh") pod "network-check-target-8992p" (UID: "a548ced1-3bed-4d35-a80d-8aa03e106398") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 15:46:25.294344 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:25.294305 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:25.294810 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.294471 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:25.294810 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.294529 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:26.294510474 +0000 UTC m=+35.568942535 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:25.395093 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:25.395067 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:25.395243 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.395224 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:25.395337 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:25.395313 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:26.395291982 +0000 UTC m=+35.669724026 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:25.433087 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:25.433047 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerStarted","Data":"0e57a5944822cd3f2852ecfa513baa64f7f2298c4bdef3f193d321026c6aa42c"} Jun 02 15:46:26.252821 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.252781 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:26.253003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.252830 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:26.253003 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.252908 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:26.255857 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.255840 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 15:46:26.256074 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.256059 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 15:46:26.256813 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.256795 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wnhfh\"" Jun 02 15:46:26.256904 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.256886 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 15:46:26.256964 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.256906 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4lmvm\"" Jun 02 15:46:26.256964 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.256886 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 15:46:26.302492 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.302458 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:26.302823 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:26.302597 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:26.302823 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:26.302656 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:28.302640401 +0000 UTC m=+37.577072444 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:26.402832 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.402796 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:26.402944 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:26.402905 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:26.403007 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:26.402989 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:28.402973535 +0000 UTC m=+37.677405579 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:26.437430 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.437398 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="0e57a5944822cd3f2852ecfa513baa64f7f2298c4bdef3f193d321026c6aa42c" exitCode=0 Jun 02 15:46:26.437560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:26.437444 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"0e57a5944822cd3f2852ecfa513baa64f7f2298c4bdef3f193d321026c6aa42c"} Jun 02 15:46:27.441615 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:27.441576 2581 generic.go:358] "Generic (PLEG): container finished" podID="ea2e3887-3d5e-4848-9389-595335ede747" containerID="779146a7e733ed5d2078195c4808b12f41e0f147fd466d8ad78e3309256c5b60" exitCode=0 Jun 02 15:46:27.442126 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:27.441619 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerDied","Data":"779146a7e733ed5d2078195c4808b12f41e0f147fd466d8ad78e3309256c5b60"} Jun 02 15:46:28.316447 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:28.316417 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:28.316598 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:28.316577 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:28.316654 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:28.316645 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:32.316630436 +0000 UTC m=+41.591062479 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:28.417641 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:28.417606 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:28.417785 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:28.417731 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:28.417785 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:28.417780 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:32.417767665 +0000 UTC m=+41.692199711 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:28.446309 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:28.446278 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-w896z" event={"ID":"ea2e3887-3d5e-4848-9389-595335ede747","Type":"ContainerStarted","Data":"4250eb20b3f1cfef4ef21a729c8057c0037dabdb6a207f5a84430b83f850912e"} Jun 02 15:46:28.469869 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:28.469822 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-w896z" podStartSLOduration=5.996330613 podStartE2EDuration="37.4698091s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:45:53.772318877 +0000 UTC m=+3.046750920" lastFinishedPulling="2026-06-02 15:46:25.245797347 +0000 UTC m=+34.520229407" observedRunningTime="2026-06-02 15:46:28.468952461 +0000 UTC m=+37.743384526" watchObservedRunningTime="2026-06-02 15:46:28.4698091 +0000 UTC m=+37.744241163" Jun 02 15:46:32.345689 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:32.345492 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:32.346080 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:32.345647 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:32.346080 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:32.345791 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:40.345776181 +0000 UTC m=+49.620208225 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:32.446275 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:32.446229 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:32.446459 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:32.446432 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:32.446522 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:32.446514 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:40.446490397 +0000 UTC m=+49.720922473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:33.756800 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.756765 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d"] Jun 02 15:46:33.759434 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.759419 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:33.761826 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.761799 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-jtp85\"" Jun 02 15:46:33.761959 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.761849 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 02 15:46:33.761959 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.761810 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 02 15:46:33.774170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.774146 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d"] Jun 02 15:46:33.855605 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.855567 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:33.855773 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.855630 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:33.956278 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.956243 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:33.956356 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.956339 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:33.956417 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:33.956397 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:46:33.956471 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:33.956462 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert podName:ce0622ed-e333-4d1a-a4ff-becd3bfa4249 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:34.456449538 +0000 UTC m=+43.730881581 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-lbf7d" (UID: "ce0622ed-e333-4d1a-a4ff-becd3bfa4249") : secret "networking-console-plugin-cert" not found Jun 02 15:46:33.956974 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:33.956954 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:34.460916 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:34.460879 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:34.461095 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:34.461013 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:46:34.461095 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:34.461068 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert podName:ce0622ed-e333-4d1a-a4ff-becd3bfa4249 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:35.461054017 +0000 UTC m=+44.735486060 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-lbf7d" (UID: "ce0622ed-e333-4d1a-a4ff-becd3bfa4249") : secret "networking-console-plugin-cert" not found Jun 02 15:46:35.057986 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.057946 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh"] Jun 02 15:46:35.060603 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.060587 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" Jun 02 15:46:35.063313 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.063290 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\"" Jun 02 15:46:35.063429 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.063294 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\"" Jun 02 15:46:35.064062 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.064041 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-5d9hh\"" Jun 02 15:46:35.071963 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.071943 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh"] Jun 02 15:46:35.164809 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.164779 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wlvpb\" (UniqueName: \"kubernetes.io/projected/756c6be7-258a-4fa4-a371-44771f9a4529-kube-api-access-wlvpb\") pod \"migrator-96c6f749-6gwvh\" (UID: \"756c6be7-258a-4fa4-a371-44771f9a4529\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" Jun 02 15:46:35.265961 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.265924 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wlvpb\" (UniqueName: \"kubernetes.io/projected/756c6be7-258a-4fa4-a371-44771f9a4529-kube-api-access-wlvpb\") pod \"migrator-96c6f749-6gwvh\" (UID: \"756c6be7-258a-4fa4-a371-44771f9a4529\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" Jun 02 15:46:35.275102 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.275071 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wlvpb\" (UniqueName: \"kubernetes.io/projected/756c6be7-258a-4fa4-a371-44771f9a4529-kube-api-access-wlvpb\") pod \"migrator-96c6f749-6gwvh\" (UID: \"756c6be7-258a-4fa4-a371-44771f9a4529\") " pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" Jun 02 15:46:35.369376 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.369288 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" Jun 02 15:46:35.466994 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.466954 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:35.467175 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:35.467119 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:46:35.467239 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:35.467198 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert podName:ce0622ed-e333-4d1a-a4ff-becd3bfa4249 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:37.467176875 +0000 UTC m=+46.741608925 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-lbf7d" (UID: "ce0622ed-e333-4d1a-a4ff-becd3bfa4249") : secret "networking-console-plugin-cert" not found Jun 02 15:46:35.502559 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:35.502525 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh"] Jun 02 15:46:35.506335 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:35.506310 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod756c6be7_258a_4fa4_a371_44771f9a4529.slice/crio-fab0f8bca893992bb5a2db49029eca485027a61113d6cd82ce69c28c892c329b WatchSource:0}: Error finding container fab0f8bca893992bb5a2db49029eca485027a61113d6cd82ce69c28c892c329b: Status 404 returned error can't find the container with id fab0f8bca893992bb5a2db49029eca485027a61113d6cd82ce69c28c892c329b Jun 02 15:46:36.462674 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:36.462633 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" event={"ID":"756c6be7-258a-4fa4-a371-44771f9a4529","Type":"ContainerStarted","Data":"fab0f8bca893992bb5a2db49029eca485027a61113d6cd82ce69c28c892c329b"} Jun 02 15:46:36.752647 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:36.752574 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p7f6j_2dad6529-26f1-4882-adc3-f3e371520b17/dns-node-resolver/0.log" Jun 02 15:46:37.465483 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:37.465311 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" event={"ID":"756c6be7-258a-4fa4-a371-44771f9a4529","Type":"ContainerStarted","Data":"322893210eed055641e3bea36ae33d5279e7a4e0bf53f3bf5293a0996a9c5031"} Jun 02 15:46:37.480250 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:37.480229 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:37.480370 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:37.480357 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:46:37.480409 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:37.480402 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert podName:ce0622ed-e333-4d1a-a4ff-becd3bfa4249 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:41.480389616 +0000 UTC m=+50.754821659 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-lbf7d" (UID: "ce0622ed-e333-4d1a-a4ff-becd3bfa4249") : secret "networking-console-plugin-cert" not found Jun 02 15:46:37.751847 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:37.751821 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8t48_ebf604f8-64aa-471f-b305-bcec9fc03d7c/node-ca/0.log" Jun 02 15:46:38.469793 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:38.468853 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" event={"ID":"756c6be7-258a-4fa4-a371-44771f9a4529","Type":"ContainerStarted","Data":"0d41c82b4bca3416f9b50d5ed9897af68fc7063846220c4eb25c5dd1f8dc851f"} Jun 02 15:46:38.486995 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:38.486949 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-96c6f749-6gwvh" podStartSLOduration=1.65744266 podStartE2EDuration="3.486935162s" podCreationTimestamp="2026-06-02 15:46:35 +0000 UTC" firstStartedPulling="2026-06-02 15:46:35.50824756 +0000 UTC m=+44.782679606" lastFinishedPulling="2026-06-02 15:46:37.337740062 +0000 UTC m=+46.612172108" observedRunningTime="2026-06-02 15:46:38.48567823 +0000 UTC m=+47.760110296" watchObservedRunningTime="2026-06-02 15:46:38.486935162 +0000 UTC m=+47.761367227" Jun 02 15:46:40.403593 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:40.403551 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:40.404038 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:40.403710 2581 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 15:46:40.404038 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:40.403771 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert podName:94c86a42-2dd5-4552-8585-01550f4f9736 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:56.403754582 +0000 UTC m=+65.678186631 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert") pod "ingress-canary-fs7v5" (UID: "94c86a42-2dd5-4552-8585-01550f4f9736") : secret "canary-serving-cert" not found Jun 02 15:46:40.504018 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:40.503982 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:40.504183 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:40.504100 2581 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 15:46:40.504183 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:40.504149 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls podName:5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:56.504136487 +0000 UTC m=+65.778568530 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls") pod "dns-default-5kpc8" (UID: "5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4") : secret "dns-default-metrics-tls" not found Jun 02 15:46:41.513405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:41.513358 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:41.513882 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:41.513517 2581 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 15:46:41.513882 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:46:41.513582 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert podName:ce0622ed-e333-4d1a-a4ff-becd3bfa4249 nodeName:}" failed. No retries permitted until 2026-06-02 15:46:49.513566242 +0000 UTC m=+58.787998284 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-lbf7d" (UID: "ce0622ed-e333-4d1a-a4ff-becd3bfa4249") : secret "networking-console-plugin-cert" not found Jun 02 15:46:49.574475 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:49.574416 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:49.578289 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:49.578249 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/ce0622ed-e333-4d1a-a4ff-becd3bfa4249-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-lbf7d\" (UID: \"ce0622ed-e333-4d1a-a4ff-becd3bfa4249\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:49.667357 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:49.667319 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" Jun 02 15:46:49.797655 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:49.797625 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d"] Jun 02 15:46:49.800931 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:49.800890 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podce0622ed_e333_4d1a_a4ff_becd3bfa4249.slice/crio-6930f2bdae71923b69f3a2ae44da24f6468622ebf5d0a911ab0d3019b6710423 WatchSource:0}: Error finding container 6930f2bdae71923b69f3a2ae44da24f6468622ebf5d0a911ab0d3019b6710423: Status 404 returned error can't find the container with id 6930f2bdae71923b69f3a2ae44da24f6468622ebf5d0a911ab0d3019b6710423 Jun 02 15:46:50.432283 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:50.432244 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-vfcmx" Jun 02 15:46:50.490907 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:50.490867 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" event={"ID":"ce0622ed-e333-4d1a-a4ff-becd3bfa4249","Type":"ContainerStarted","Data":"6930f2bdae71923b69f3a2ae44da24f6468622ebf5d0a911ab0d3019b6710423"} Jun 02 15:46:51.494533 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:51.494445 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" event={"ID":"ce0622ed-e333-4d1a-a4ff-becd3bfa4249","Type":"ContainerStarted","Data":"343439b161fd1f4c5a12c378816eb9f24882e683f299b853c0c4176e3b2b8d94"} Jun 02 15:46:51.512692 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:51.512641 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-lbf7d" podStartSLOduration=17.073044752 podStartE2EDuration="18.512627164s" podCreationTimestamp="2026-06-02 15:46:33 +0000 UTC" firstStartedPulling="2026-06-02 15:46:49.802674215 +0000 UTC m=+59.077106258" lastFinishedPulling="2026-06-02 15:46:51.242256612 +0000 UTC m=+60.516688670" observedRunningTime="2026-06-02 15:46:51.511891592 +0000 UTC m=+60.786323657" watchObservedRunningTime="2026-06-02 15:46:51.512627164 +0000 UTC m=+60.787059229" Jun 02 15:46:56.422247 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.422203 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:56.424711 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.424691 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/94c86a42-2dd5-4552-8585-01550f4f9736-cert\") pod \"ingress-canary-fs7v5\" (UID: \"94c86a42-2dd5-4552-8585-01550f4f9736\") " pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:56.445384 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.445360 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-s8ml8\"" Jun 02 15:46:56.453666 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.453645 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-fs7v5" Jun 02 15:46:56.528121 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.527594 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:56.530726 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.530699 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4-metrics-tls\") pod \"dns-default-5kpc8\" (UID: \"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4\") " pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:56.594557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.594522 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-fs7v5"] Jun 02 15:46:56.598369 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:56.598339 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod94c86a42_2dd5_4552_8585_01550f4f9736.slice/crio-7e3f8c8bf49fe42a9d572dda6d8d338fdde979b0a26bc03cb6727dfd270c23dc WatchSource:0}: Error finding container 7e3f8c8bf49fe42a9d572dda6d8d338fdde979b0a26bc03cb6727dfd270c23dc: Status 404 returned error can't find the container with id 7e3f8c8bf49fe42a9d572dda6d8d338fdde979b0a26bc03cb6727dfd270c23dc Jun 02 15:46:56.750957 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.750929 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-cpsk2\"" Jun 02 15:46:56.758721 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.758702 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:56.885057 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.885029 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-5kpc8"] Jun 02 15:46:56.888044 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:56.888006 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5c7ef390_f64f_4a10_9aa6_d9fc78a36fc4.slice/crio-dcdc5da67bf4d4c13488cbd293111a4f43e6bafa8047ea1e144e7667033e0cce WatchSource:0}: Error finding container dcdc5da67bf4d4c13488cbd293111a4f43e6bafa8047ea1e144e7667033e0cce: Status 404 returned error can't find the container with id dcdc5da67bf4d4c13488cbd293111a4f43e6bafa8047ea1e144e7667033e0cce Jun 02 15:46:56.931349 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.931320 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:56.937253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.937233 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 15:46:56.944533 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:56.944511 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/8f9d8e73-83df-4549-b1a1-5c23188d866d-metrics-certs\") pod \"network-metrics-daemon-rjbmw\" (UID: \"8f9d8e73-83df-4549-b1a1-5c23188d866d\") " pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:57.032052 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.031955 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:57.034737 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.034718 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 15:46:57.045488 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.045465 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/ff60fcd6-3fa3-4be1-b821-fdebcadc7675-original-pull-secret\") pod \"global-pull-secret-syncer-tfkkt\" (UID: \"ff60fcd6-3fa3-4be1-b821-fdebcadc7675\") " pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:57.133079 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.133045 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:57.135914 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.135888 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 15:46:57.145960 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.145934 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 15:46:57.156660 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.156636 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lkhfh\" (UniqueName: \"kubernetes.io/projected/a548ced1-3bed-4d35-a80d-8aa03e106398-kube-api-access-lkhfh\") pod \"network-check-target-8992p\" (UID: \"a548ced1-3bed-4d35-a80d-8aa03e106398\") " pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:57.162204 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.162184 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-tfkkt" Jun 02 15:46:57.169235 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.169209 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-wnhfh\"" Jun 02 15:46:57.174420 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.174396 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-4lmvm\"" Jun 02 15:46:57.177330 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.177310 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-rjbmw" Jun 02 15:46:57.183054 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.183034 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:46:57.327105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.327026 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-tfkkt"] Jun 02 15:46:57.335701 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:57.335658 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff60fcd6_3fa3_4be1_b821_fdebcadc7675.slice/crio-da2a162f2bdd71aabd6adc795c64a783f8530f63dfc75a6e3ad33c906bdde072 WatchSource:0}: Error finding container da2a162f2bdd71aabd6adc795c64a783f8530f63dfc75a6e3ad33c906bdde072: Status 404 returned error can't find the container with id da2a162f2bdd71aabd6adc795c64a783f8530f63dfc75a6e3ad33c906bdde072 Jun 02 15:46:57.350497 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.350466 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-rjbmw"] Jun 02 15:46:57.356331 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:57.356237 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f9d8e73_83df_4549_b1a1_5c23188d866d.slice/crio-f0f719cd306cb3534db216b417570df35d3b7ed8d82c44c10f384fb39de5ba03 WatchSource:0}: Error finding container f0f719cd306cb3534db216b417570df35d3b7ed8d82c44c10f384fb39de5ba03: Status 404 returned error can't find the container with id f0f719cd306cb3534db216b417570df35d3b7ed8d82c44c10f384fb39de5ba03 Jun 02 15:46:57.367739 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.367712 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-8992p"] Jun 02 15:46:57.371110 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:46:57.371079 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda548ced1_3bed_4d35_a80d_8aa03e106398.slice/crio-ba1277b436c23f1fb41708c0157aee0f98a314fb0db97fa51fe4f75573527ddd WatchSource:0}: Error finding container ba1277b436c23f1fb41708c0157aee0f98a314fb0db97fa51fe4f75573527ddd: Status 404 returned error can't find the container with id ba1277b436c23f1fb41708c0157aee0f98a314fb0db97fa51fe4f75573527ddd Jun 02 15:46:57.519253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.519204 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjbmw" event={"ID":"8f9d8e73-83df-4549-b1a1-5c23188d866d","Type":"ContainerStarted","Data":"f0f719cd306cb3534db216b417570df35d3b7ed8d82c44c10f384fb39de5ba03"} Jun 02 15:46:57.520552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.520516 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfkkt" event={"ID":"ff60fcd6-3fa3-4be1-b821-fdebcadc7675","Type":"ContainerStarted","Data":"da2a162f2bdd71aabd6adc795c64a783f8530f63dfc75a6e3ad33c906bdde072"} Jun 02 15:46:57.521753 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.521715 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8992p" event={"ID":"a548ced1-3bed-4d35-a80d-8aa03e106398","Type":"ContainerStarted","Data":"ba1277b436c23f1fb41708c0157aee0f98a314fb0db97fa51fe4f75573527ddd"} Jun 02 15:46:57.523221 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.523196 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5kpc8" event={"ID":"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4","Type":"ContainerStarted","Data":"dcdc5da67bf4d4c13488cbd293111a4f43e6bafa8047ea1e144e7667033e0cce"} Jun 02 15:46:57.524612 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:57.524590 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fs7v5" event={"ID":"94c86a42-2dd5-4552-8585-01550f4f9736","Type":"ContainerStarted","Data":"7e3f8c8bf49fe42a9d572dda6d8d338fdde979b0a26bc03cb6727dfd270c23dc"} Jun 02 15:46:59.535335 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.535293 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjbmw" event={"ID":"8f9d8e73-83df-4549-b1a1-5c23188d866d","Type":"ContainerStarted","Data":"51101c559e964d680d5477943ed22f796f93f3fdf4eb7c242040248bf4e2eb0e"} Jun 02 15:46:59.541240 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.541199 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5kpc8" event={"ID":"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4","Type":"ContainerStarted","Data":"7b8375e565ade6803b42cb4ebf5d38c885f5361692d8e2159387bd5dc0effe10"} Jun 02 15:46:59.541577 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.541539 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-5kpc8" Jun 02 15:46:59.543930 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.543492 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-fs7v5" event={"ID":"94c86a42-2dd5-4552-8585-01550f4f9736","Type":"ContainerStarted","Data":"ae3a5fabdb06ca7c4489a93d03a98eecfbb93a7dc64be86ab30832eb1d7b1053"} Jun 02 15:46:59.556954 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.556382 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-rjbmw" podStartSLOduration=66.682099539 podStartE2EDuration="1m8.556362428s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.358937588 +0000 UTC m=+66.633369635" lastFinishedPulling="2026-06-02 15:46:59.233200478 +0000 UTC m=+68.507632524" observedRunningTime="2026-06-02 15:46:59.555556335 +0000 UTC m=+68.829988401" watchObservedRunningTime="2026-06-02 15:46:59.556362428 +0000 UTC m=+68.830794495" Jun 02 15:46:59.576307 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.576081 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-5kpc8" podStartSLOduration=33.229773454 podStartE2EDuration="35.576063401s" podCreationTimestamp="2026-06-02 15:46:24 +0000 UTC" firstStartedPulling="2026-06-02 15:46:56.891864769 +0000 UTC m=+66.166296812" lastFinishedPulling="2026-06-02 15:46:59.238154709 +0000 UTC m=+68.512586759" observedRunningTime="2026-06-02 15:46:59.57443432 +0000 UTC m=+68.848872963" watchObservedRunningTime="2026-06-02 15:46:59.576063401 +0000 UTC m=+68.850495467" Jun 02 15:46:59.591911 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:46:59.591567 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-fs7v5" podStartSLOduration=32.958271064 podStartE2EDuration="35.591552959s" podCreationTimestamp="2026-06-02 15:46:24 +0000 UTC" firstStartedPulling="2026-06-02 15:46:56.600729232 +0000 UTC m=+65.875161275" lastFinishedPulling="2026-06-02 15:46:59.234011113 +0000 UTC m=+68.508443170" observedRunningTime="2026-06-02 15:46:59.590658973 +0000 UTC m=+68.865091038" watchObservedRunningTime="2026-06-02 15:46:59.591552959 +0000 UTC m=+68.865985018" Jun 02 15:47:00.550457 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:00.550419 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-5kpc8" event={"ID":"5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4","Type":"ContainerStarted","Data":"3e6d9b53485053cc755b731b3d082cdc54916771d16e8a144331278102cf9ff8"} Jun 02 15:47:00.552096 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:00.552067 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-rjbmw" event={"ID":"8f9d8e73-83df-4549-b1a1-5c23188d866d","Type":"ContainerStarted","Data":"efd659a5b3cad1de85af1d21454e9e025f12e1d667cd9f8ac3e406082718ce37"} Jun 02 15:47:01.210872 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.210839 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-rcdrm"] Jun 02 15:47:01.214515 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.214481 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.216944 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.216915 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 15:47:01.218036 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.218009 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hqmcq\"" Jun 02 15:47:01.218286 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.218250 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 15:47:01.218393 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.218330 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 15:47:01.218614 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.218595 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 15:47:01.227345 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.226894 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rcdrm"] Jun 02 15:47:01.261561 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.261525 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caa710a6-f825-44d2-96a4-d84e15cb7e52-data-volume\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.261723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.261576 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.261723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.261605 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caa710a6-f825-44d2-96a4-d84e15cb7e52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.261723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.261634 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caa710a6-f825-44d2-96a4-d84e15cb7e52-crio-socket\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.261723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.261662 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xjxhv\" (UniqueName: \"kubernetes.io/projected/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-api-access-xjxhv\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.310169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.310136 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c65b8dc4b-pjz5q"] Jun 02 15:47:01.311991 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.311970 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:01.314789 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.314765 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-mlpx8\"" Jun 02 15:47:01.315050 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.315027 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\"" Jun 02 15:47:01.315146 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.315027 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\"" Jun 02 15:47:01.324847 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.324822 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-pjz5q"] Jun 02 15:47:01.362389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362349 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgrrt\" (UniqueName: \"kubernetes.io/projected/2bac1ebf-7eed-458d-ac1d-05d4abca2855-kube-api-access-cgrrt\") pod \"downloads-c65b8dc4b-pjz5q\" (UID: \"2bac1ebf-7eed-458d-ac1d-05d4abca2855\") " pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:01.362389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362395 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caa710a6-f825-44d2-96a4-d84e15cb7e52-data-volume\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362617 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362436 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362617 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362460 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caa710a6-f825-44d2-96a4-d84e15cb7e52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362617 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362485 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caa710a6-f825-44d2-96a4-d84e15cb7e52-crio-socket\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362617 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362540 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xjxhv\" (UniqueName: \"kubernetes.io/projected/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-api-access-xjxhv\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362657 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/caa710a6-f825-44d2-96a4-d84e15cb7e52-crio-socket\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.362815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.362804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/caa710a6-f825-44d2-96a4-d84e15cb7e52-data-volume\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.363139 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.363117 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.365545 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.365524 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/caa710a6-f825-44d2-96a4-d84e15cb7e52-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.377779 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.377753 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xjxhv\" (UniqueName: \"kubernetes.io/projected/caa710a6-f825-44d2-96a4-d84e15cb7e52-kube-api-access-xjxhv\") pod \"insights-runtime-extractor-rcdrm\" (UID: \"caa710a6-f825-44d2-96a4-d84e15cb7e52\") " pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.463632 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.463536 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cgrrt\" (UniqueName: \"kubernetes.io/projected/2bac1ebf-7eed-458d-ac1d-05d4abca2855-kube-api-access-cgrrt\") pod \"downloads-c65b8dc4b-pjz5q\" (UID: \"2bac1ebf-7eed-458d-ac1d-05d4abca2855\") " pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:01.475732 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.475703 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgrrt\" (UniqueName: \"kubernetes.io/projected/2bac1ebf-7eed-458d-ac1d-05d4abca2855-kube-api-access-cgrrt\") pod \"downloads-c65b8dc4b-pjz5q\" (UID: \"2bac1ebf-7eed-458d-ac1d-05d4abca2855\") " pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:01.527052 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.527007 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-rcdrm" Jun 02 15:47:01.621701 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:01.621635 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:02.631433 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:02.630937 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c65b8dc4b-pjz5q"] Jun 02 15:47:02.634883 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:02.634857 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2bac1ebf_7eed_458d_ac1d_05d4abca2855.slice/crio-bf473170d900e2067c5dbac5c204e9c248e20151c3116eea57d3d3df9051149c WatchSource:0}: Error finding container bf473170d900e2067c5dbac5c204e9c248e20151c3116eea57d3d3df9051149c: Status 404 returned error can't find the container with id bf473170d900e2067c5dbac5c204e9c248e20151c3116eea57d3d3df9051149c Jun 02 15:47:02.645987 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:02.645959 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-rcdrm"] Jun 02 15:47:02.648946 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:02.648919 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcaa710a6_f825_44d2_96a4_d84e15cb7e52.slice/crio-afe3019661952e28b04cb3492eaa1088c04b8420105975df7aa3c8f1cbcd94a3 WatchSource:0}: Error finding container afe3019661952e28b04cb3492eaa1088c04b8420105975df7aa3c8f1cbcd94a3: Status 404 returned error can't find the container with id afe3019661952e28b04cb3492eaa1088c04b8420105975df7aa3c8f1cbcd94a3 Jun 02 15:47:03.571187 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.571098 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-tfkkt" event={"ID":"ff60fcd6-3fa3-4be1-b821-fdebcadc7675","Type":"ContainerStarted","Data":"3b85a13ad3c6308da00afb62e5eea1ba1ce3882bd856007f92ed0f38a9ae2392"} Jun 02 15:47:03.572701 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.572675 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-8992p" event={"ID":"a548ced1-3bed-4d35-a80d-8aa03e106398","Type":"ContainerStarted","Data":"0f8ffc388fb7db2688dd7f6ca5e9b7cfca16d9f3216ffdc3d2ecf7f24f363dcf"} Jun 02 15:47:03.572829 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.572816 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:47:03.573697 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.573674 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-pjz5q" event={"ID":"2bac1ebf-7eed-458d-ac1d-05d4abca2855","Type":"ContainerStarted","Data":"bf473170d900e2067c5dbac5c204e9c248e20151c3116eea57d3d3df9051149c"} Jun 02 15:47:03.575246 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.575225 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcdrm" event={"ID":"caa710a6-f825-44d2-96a4-d84e15cb7e52","Type":"ContainerStarted","Data":"37490d025d9d8bbb50e6e59243c790026f6441c888c1995b1bb1ed0acf62ff7b"} Jun 02 15:47:03.575354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.575252 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcdrm" event={"ID":"caa710a6-f825-44d2-96a4-d84e15cb7e52","Type":"ContainerStarted","Data":"f41a756ab1c50cf6de1189f64390ae6cecde30438d2046c668f51091cec63b36"} Jun 02 15:47:03.575354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.575282 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcdrm" event={"ID":"caa710a6-f825-44d2-96a4-d84e15cb7e52","Type":"ContainerStarted","Data":"afe3019661952e28b04cb3492eaa1088c04b8420105975df7aa3c8f1cbcd94a3"} Jun 02 15:47:03.605839 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.605281 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-tfkkt" podStartSLOduration=66.451727029 podStartE2EDuration="1m11.605236294s" podCreationTimestamp="2026-06-02 15:45:52 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.337577971 +0000 UTC m=+66.612010015" lastFinishedPulling="2026-06-02 15:47:02.491087233 +0000 UTC m=+71.765519280" observedRunningTime="2026-06-02 15:47:03.588807198 +0000 UTC m=+72.863239263" watchObservedRunningTime="2026-06-02 15:47:03.605236294 +0000 UTC m=+72.879668341" Jun 02 15:47:03.606767 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:03.606714 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-8992p" podStartSLOduration=67.502620968 podStartE2EDuration="1m12.606696903s" podCreationTimestamp="2026-06-02 15:45:51 +0000 UTC" firstStartedPulling="2026-06-02 15:46:57.373322352 +0000 UTC m=+66.647754410" lastFinishedPulling="2026-06-02 15:47:02.477398284 +0000 UTC m=+71.751830345" observedRunningTime="2026-06-02 15:47:03.604844202 +0000 UTC m=+72.879276268" watchObservedRunningTime="2026-06-02 15:47:03.606696903 +0000 UTC m=+72.881128968" Jun 02 15:47:05.584157 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:05.584033 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-rcdrm" event={"ID":"caa710a6-f825-44d2-96a4-d84e15cb7e52","Type":"ContainerStarted","Data":"2ee81844287e16038aff9d292aca7c4d7416342f8979fdabe852187deb0408df"} Jun 02 15:47:05.603678 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:05.603611 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-rcdrm" podStartSLOduration=2.6412786 podStartE2EDuration="4.603593783s" podCreationTimestamp="2026-06-02 15:47:01 +0000 UTC" firstStartedPulling="2026-06-02 15:47:02.701555868 +0000 UTC m=+71.975987913" lastFinishedPulling="2026-06-02 15:47:04.663871051 +0000 UTC m=+73.938303096" observedRunningTime="2026-06-02 15:47:05.602644744 +0000 UTC m=+74.877076809" watchObservedRunningTime="2026-06-02 15:47:05.603593783 +0000 UTC m=+74.878025849" Jun 02 15:47:10.557904 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:10.557864 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-5kpc8" Jun 02 15:47:20.425873 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.425831 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:47:20.430845 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.430821 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.446723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.446695 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\"" Jun 02 15:47:20.446723 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.446695 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\"" Jun 02 15:47:20.447674 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.447620 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-snv8w\"" Jun 02 15:47:20.447674 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.447647 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\"" Jun 02 15:47:20.447863 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.447654 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\"" Jun 02 15:47:20.447863 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.447760 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\"" Jun 02 15:47:20.473665 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.473627 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:47:20.496366 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496336 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.496548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bvgk7\" (UniqueName: \"kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.496548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496434 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.496548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496458 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.496548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496515 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.496704 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.496586 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597088 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597051 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597099 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597130 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597169 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597203 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.597253 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597228 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bvgk7\" (UniqueName: \"kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.598021 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.597969 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.598157 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.598095 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.598460 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.598203 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.600088 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.600063 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.600234 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.600213 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.618167 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.618138 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bvgk7\" (UniqueName: \"kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7\") pod \"console-56d49c9f68-4tggz\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:20.740255 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:20.740153 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:21.017924 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.017902 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:47:21.021690 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:21.021663 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda3983390_16b7_4f63_8d8e_2631ba692487.slice/crio-96148f4dc60380eca00ad3bb06a5fb32ac66199296e61e628b771adf7f650c4c WatchSource:0}: Error finding container 96148f4dc60380eca00ad3bb06a5fb32ac66199296e61e628b771adf7f650c4c: Status 404 returned error can't find the container with id 96148f4dc60380eca00ad3bb06a5fb32ac66199296e61e628b771adf7f650c4c Jun 02 15:47:21.639891 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.639843 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c65b8dc4b-pjz5q" event={"ID":"2bac1ebf-7eed-458d-ac1d-05d4abca2855","Type":"ContainerStarted","Data":"ab1c992712a64fa34de0cb2a4e756b506d45ffa8c9ff3e3dc4a8cd251655c6bd"} Jun 02 15:47:21.640594 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.640570 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:21.642467 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.642426 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d49c9f68-4tggz" event={"ID":"a3983390-16b7-4f63-8d8e-2631ba692487","Type":"ContainerStarted","Data":"96148f4dc60380eca00ad3bb06a5fb32ac66199296e61e628b771adf7f650c4c"} Jun 02 15:47:21.655573 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.655530 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c65b8dc4b-pjz5q" Jun 02 15:47:21.677821 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:21.677321 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c65b8dc4b-pjz5q" podStartSLOduration=2.338100754 podStartE2EDuration="20.677304082s" podCreationTimestamp="2026-06-02 15:47:01 +0000 UTC" firstStartedPulling="2026-06-02 15:47:02.637542784 +0000 UTC m=+71.911974836" lastFinishedPulling="2026-06-02 15:47:20.976746103 +0000 UTC m=+90.251178164" observedRunningTime="2026-06-02 15:47:21.674663671 +0000 UTC m=+90.949095749" watchObservedRunningTime="2026-06-02 15:47:21.677304082 +0000 UTC m=+90.951736148" Jun 02 15:47:22.429670 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.429606 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:47:22.447863 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.447831 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.459391 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.459360 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:47:22.463597 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.463563 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\"" Jun 02 15:47:22.511689 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511646 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.511894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511705 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.511894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511734 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.511894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511796 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d94gv\" (UniqueName: \"kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.511894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511849 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.511894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511884 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.512085 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.511921 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.612870 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.612833 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.612886 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.612913 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.612952 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d94gv\" (UniqueName: \"kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.612996 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.613038 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.613078 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613899 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.613730 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.613899 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.613868 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.614067 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.614045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.614125 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.614045 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.615915 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.615892 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.616390 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.616366 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.623999 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.623973 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d94gv\" (UniqueName: \"kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv\") pod \"console-67dc7f488f-spld6\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.761215 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.761125 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:22.971239 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:22.971181 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:47:22.978705 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:22.978666 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1eca3b80_b5c7_4fd3_b761_d5b00e18b17d.slice/crio-de5b8547eb79477433df99813c13748ff2085f9d8c8e3fe72a3637927fd94b39 WatchSource:0}: Error finding container de5b8547eb79477433df99813c13748ff2085f9d8c8e3fe72a3637927fd94b39: Status 404 returned error can't find the container with id de5b8547eb79477433df99813c13748ff2085f9d8c8e3fe72a3637927fd94b39 Jun 02 15:47:23.651112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:23.651047 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dc7f488f-spld6" event={"ID":"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d","Type":"ContainerStarted","Data":"de5b8547eb79477433df99813c13748ff2085f9d8c8e3fe72a3637927fd94b39"} Jun 02 15:47:26.232987 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.232949 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq"] Jun 02 15:47:26.244631 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.244606 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.248682 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.248656 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\"" Jun 02 15:47:26.249701 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.249676 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\"" Jun 02 15:47:26.249820 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.249755 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 15:47:26.249820 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.249781 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 15:47:26.249942 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.249825 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-t82zb\"" Jun 02 15:47:26.249942 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.249755 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 15:47:26.257652 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.257628 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq"] Jun 02 15:47:26.348838 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.348798 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cbxlz\" (UniqueName: \"kubernetes.io/projected/1ecef6fd-3ef8-4de2-8146-f930d50696f7-kube-api-access-cbxlz\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.349045 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.348862 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.349045 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.348888 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.349045 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.348925 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecef6fd-3ef8-4de2-8146-f930d50696f7-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.450118 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.450075 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cbxlz\" (UniqueName: \"kubernetes.io/projected/1ecef6fd-3ef8-4de2-8146-f930d50696f7-kube-api-access-cbxlz\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.450378 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.450136 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.450378 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.450166 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.450378 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.450207 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecef6fd-3ef8-4de2-8146-f930d50696f7-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.450378 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:26.450347 2581 secret.go:189] Couldn't get secret openshift-monitoring/prometheus-operator-tls: secret "prometheus-operator-tls" not found Jun 02 15:47:26.450586 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:26.450427 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls podName:1ecef6fd-3ef8-4de2-8146-f930d50696f7 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:26.950405455 +0000 UTC m=+96.224837507 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "prometheus-operator-tls" (UniqueName: "kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls") pod "prometheus-operator-658bcd8bdf-dzscq" (UID: "1ecef6fd-3ef8-4de2-8146-f930d50696f7") : secret "prometheus-operator-tls" not found Jun 02 15:47:26.451037 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.451017 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1ecef6fd-3ef8-4de2-8146-f930d50696f7-metrics-client-ca\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.453168 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.453143 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.464853 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.464824 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cbxlz\" (UniqueName: \"kubernetes.io/projected/1ecef6fd-3ef8-4de2-8146-f930d50696f7-kube-api-access-cbxlz\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.662785 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.662751 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d49c9f68-4tggz" event={"ID":"a3983390-16b7-4f63-8d8e-2631ba692487","Type":"ContainerStarted","Data":"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe"} Jun 02 15:47:26.664406 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.664372 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dc7f488f-spld6" event={"ID":"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d","Type":"ContainerStarted","Data":"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb"} Jun 02 15:47:26.694539 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.694472 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-56d49c9f68-4tggz" podStartSLOduration=1.74014983 podStartE2EDuration="6.694452852s" podCreationTimestamp="2026-06-02 15:47:20 +0000 UTC" firstStartedPulling="2026-06-02 15:47:21.023573375 +0000 UTC m=+90.298005422" lastFinishedPulling="2026-06-02 15:47:25.977876386 +0000 UTC m=+95.252308444" observedRunningTime="2026-06-02 15:47:26.694366081 +0000 UTC m=+95.968798158" watchObservedRunningTime="2026-06-02 15:47:26.694452852 +0000 UTC m=+95.968884917" Jun 02 15:47:26.744740 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.744642 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-67dc7f488f-spld6" podStartSLOduration=1.740942933 podStartE2EDuration="4.744626158s" podCreationTimestamp="2026-06-02 15:47:22 +0000 UTC" firstStartedPulling="2026-06-02 15:47:22.981494365 +0000 UTC m=+92.255926408" lastFinishedPulling="2026-06-02 15:47:25.985177581 +0000 UTC m=+95.259609633" observedRunningTime="2026-06-02 15:47:26.742371267 +0000 UTC m=+96.016803336" watchObservedRunningTime="2026-06-02 15:47:26.744626158 +0000 UTC m=+96.019058223" Jun 02 15:47:26.954713 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.954668 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:26.957528 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:26.957492 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/1ecef6fd-3ef8-4de2-8146-f930d50696f7-prometheus-operator-tls\") pod \"prometheus-operator-658bcd8bdf-dzscq\" (UID: \"1ecef6fd-3ef8-4de2-8146-f930d50696f7\") " pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:27.156139 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:27.156050 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" Jun 02 15:47:27.359857 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:27.359828 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq"] Jun 02 15:47:27.367569 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:27.362144 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1ecef6fd_3ef8_4de2_8146_f930d50696f7.slice/crio-932cb1a7c7ce50566610e167a7c7e0c7cd530266e163081f07aea23313eea713 WatchSource:0}: Error finding container 932cb1a7c7ce50566610e167a7c7e0c7cd530266e163081f07aea23313eea713: Status 404 returned error can't find the container with id 932cb1a7c7ce50566610e167a7c7e0c7cd530266e163081f07aea23313eea713 Jun 02 15:47:27.668886 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:27.668850 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" event={"ID":"1ecef6fd-3ef8-4de2-8146-f930d50696f7","Type":"ContainerStarted","Data":"932cb1a7c7ce50566610e167a7c7e0c7cd530266e163081f07aea23313eea713"} Jun 02 15:47:29.678828 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:29.678790 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" event={"ID":"1ecef6fd-3ef8-4de2-8146-f930d50696f7","Type":"ContainerStarted","Data":"5335447b0d81c48d651c7e30bbed9f2873416b1f1d9a699b4734375802b93e19"} Jun 02 15:47:29.679188 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:29.678837 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" event={"ID":"1ecef6fd-3ef8-4de2-8146-f930d50696f7","Type":"ContainerStarted","Data":"6de35a1e4f73d6a5e0cbfaf48ed54b55579c72688010d25baf857193c928cbeb"} Jun 02 15:47:30.734823 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:30.734767 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-658bcd8bdf-dzscq" podStartSLOduration=2.6943416 podStartE2EDuration="4.734753553s" podCreationTimestamp="2026-06-02 15:47:26 +0000 UTC" firstStartedPulling="2026-06-02 15:47:27.364288488 +0000 UTC m=+96.638720539" lastFinishedPulling="2026-06-02 15:47:29.404700433 +0000 UTC m=+98.679132492" observedRunningTime="2026-06-02 15:47:30.733717766 +0000 UTC m=+100.008149836" watchObservedRunningTime="2026-06-02 15:47:30.734753553 +0000 UTC m=+100.009185619" Jun 02 15:47:30.740745 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:30.740720 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:30.741012 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:30.740988 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:30.745530 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:30.745510 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:31.688834 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:31.688806 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:47:32.762610 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:32.762573 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:32.763093 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:32.762624 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:32.767049 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:32.767029 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:32.995939 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:32.995905 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-9s4q6"] Jun 02 15:47:33.009494 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.009465 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.012372 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.012348 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 15:47:33.012593 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.012578 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 15:47:33.012684 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.012617 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-6dctl\"" Jun 02 15:47:33.012850 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.012835 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 15:47:33.108393 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108358 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vc24s\" (UniqueName: \"kubernetes.io/projected/683ef396-e7f3-4991-99c9-c1def1c25c19-kube-api-access-vc24s\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108403 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-sys\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108426 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-root\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108443 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-wtmp\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108470 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-textfile\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108499 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-metrics-client-ca\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108518 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108533 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-accelerators-collector-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.108613 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.108553 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.209763 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.209728 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-metrics-client-ca\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.209763 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.209768 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.209785 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-accelerators-collector-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.209810 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:33.209882 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.209950 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vc24s\" (UniqueName: \"kubernetes.io/projected/683ef396-e7f3-4991-99c9-c1def1c25c19-kube-api-access-vc24s\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:33.209957 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls podName:683ef396-e7f3-4991-99c9-c1def1c25c19 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:33.709935405 +0000 UTC m=+102.984367455 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls") pod "node-exporter-9s4q6" (UID: "683ef396-e7f3-4991-99c9-c1def1c25c19") : secret "node-exporter-tls" not found Jun 02 15:47:33.210038 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210021 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-sys\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210049 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-root\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210074 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-wtmp\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210104 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-textfile\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210119 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-sys\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210177 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-root\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210389 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210346 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-wtmp\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210631 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210473 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-textfile\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210631 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210480 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-metrics-client-ca\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.210631 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.210510 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-accelerators-collector-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.212411 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.212390 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.223254 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.223226 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vc24s\" (UniqueName: \"kubernetes.io/projected/683ef396-e7f3-4991-99c9-c1def1c25c19-kube-api-access-vc24s\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.693725 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.693690 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:47:33.714014 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.713985 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:33.714134 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:33.714119 2581 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 15:47:33.714218 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:33.714207 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls podName:683ef396-e7f3-4991-99c9-c1def1c25c19 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:34.71419076 +0000 UTC m=+103.988622802 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls") pod "node-exporter-9s4q6" (UID: "683ef396-e7f3-4991-99c9-c1def1c25c19") : secret "node-exporter-tls" not found Jun 02 15:47:33.755450 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:33.755421 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:47:34.582846 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:34.582813 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-8992p" Jun 02 15:47:34.723200 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:34.723166 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:34.725346 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:34.725328 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/683ef396-e7f3-4991-99c9-c1def1c25c19-node-exporter-tls\") pod \"node-exporter-9s4q6\" (UID: \"683ef396-e7f3-4991-99c9-c1def1c25c19\") " pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:34.818483 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:34.818447 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-9s4q6" Jun 02 15:47:34.826852 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:34.826824 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod683ef396_e7f3_4991_99c9_c1def1c25c19.slice/crio-012d732f2bf0995fce4d3ecea3e034e27cdb48119bb4e3778bb1b24e0ae6ae4b WatchSource:0}: Error finding container 012d732f2bf0995fce4d3ecea3e034e27cdb48119bb4e3778bb1b24e0ae6ae4b: Status 404 returned error can't find the container with id 012d732f2bf0995fce4d3ecea3e034e27cdb48119bb4e3778bb1b24e0ae6ae4b Jun 02 15:47:35.697587 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:35.697537 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9s4q6" event={"ID":"683ef396-e7f3-4991-99c9-c1def1c25c19","Type":"ContainerStarted","Data":"012d732f2bf0995fce4d3ecea3e034e27cdb48119bb4e3778bb1b24e0ae6ae4b"} Jun 02 15:47:36.701798 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:36.701759 2581 generic.go:358] "Generic (PLEG): container finished" podID="683ef396-e7f3-4991-99c9-c1def1c25c19" containerID="edbd9d1e96664a708a351ee366412f287da55ad52ee98198686c2ccdbeb56bdb" exitCode=0 Jun 02 15:47:36.702143 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:36.701850 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9s4q6" event={"ID":"683ef396-e7f3-4991-99c9-c1def1c25c19","Type":"ContainerDied","Data":"edbd9d1e96664a708a351ee366412f287da55ad52ee98198686c2ccdbeb56bdb"} Jun 02 15:47:37.384526 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.384491 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-6777557fb5-tfnls"] Jun 02 15:47:37.407998 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.407964 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6777557fb5-tfnls"] Jun 02 15:47:37.408172 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.408102 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.411029 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.411008 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\"" Jun 02 15:47:37.413329 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.413310 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\"" Jun 02 15:47:37.414195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.414176 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-x7v92\"" Jun 02 15:47:37.414357 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.414239 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\"" Jun 02 15:47:37.414432 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.414289 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\"" Jun 02 15:47:37.414432 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.414241 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-7o3ddjsnlm4q2\"" Jun 02 15:47:37.508844 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.508810 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:47:37.534803 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.534769 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq"] Jun 02 15:47:37.534963 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.534942 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.546936 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.546907 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-tls\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547016 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-metrics-server-audit-profiles\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547063 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547099 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d4f841f-616a-4743-9b42-f41fc750a150-audit-log\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547317 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547124 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-client-certs\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547317 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547205 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-client-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.547317 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.547299 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sgc8r\" (UniqueName: \"kubernetes.io/projected/7d4f841f-616a-4743-9b42-f41fc750a150-kube-api-access-sgc8r\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.551098 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.551080 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:47:37.551196 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.551102 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq"] Jun 02 15:47:37.551196 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.551190 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:37.556392 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.556369 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\"" Jun 02 15:47:37.556519 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.556375 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-mzwzd\"" Jun 02 15:47:37.648131 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648131 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648083 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sgc8r\" (UniqueName: \"kubernetes.io/projected/7d4f841f-616a-4743-9b42-f41fc750a150-kube-api-access-sgc8r\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648382 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648139 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648382 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648196 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-xtksq\" (UID: \"b478c046-985d-47ab-86e3-b729b32be572\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:37.648382 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648230 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-tls\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648382 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648250 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-metrics-server-audit-profiles\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648382 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648296 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648616 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648405 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b26k8\" (UniqueName: \"kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648616 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648431 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648616 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648454 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648616 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648498 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.648815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648619 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d4f841f-616a-4743-9b42-f41fc750a150-audit-log\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648657 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-client-certs\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648692 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-client-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.648815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648719 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.649001 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.648961 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/7d4f841f-616a-4743-9b42-f41fc750a150-audit-log\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.649227 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.649136 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.649296 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.649283 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/7d4f841f-616a-4743-9b42-f41fc750a150-metrics-server-audit-profiles\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.650976 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.650941 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-tls\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.651134 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.651117 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-secret-metrics-server-client-certs\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.651202 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.651182 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/7d4f841f-616a-4743-9b42-f41fc750a150-client-ca-bundle\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.658392 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.658363 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sgc8r\" (UniqueName: \"kubernetes.io/projected/7d4f841f-616a-4743-9b42-f41fc750a150-kube-api-access-sgc8r\") pod \"metrics-server-6777557fb5-tfnls\" (UID: \"7d4f841f-616a-4743-9b42-f41fc750a150\") " pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.705867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.705828 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9s4q6" event={"ID":"683ef396-e7f3-4991-99c9-c1def1c25c19","Type":"ContainerStarted","Data":"c97642d0d597e2b4444a5af11388615094bd3539805db964076faec8bcce857e"} Jun 02 15:47:37.705867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.705862 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-9s4q6" event={"ID":"683ef396-e7f3-4991-99c9-c1def1c25c19","Type":"ContainerStarted","Data":"d0503c3a0affe0c46ecce2fe8277536234ba5ed9ac4a79f7346999dfec50bb0a"} Jun 02 15:47:37.718402 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.718378 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:37.727597 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.727532 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-9s4q6" podStartSLOduration=4.145834781 podStartE2EDuration="5.727516364s" podCreationTimestamp="2026-06-02 15:47:32 +0000 UTC" firstStartedPulling="2026-06-02 15:47:34.828432968 +0000 UTC m=+104.102865013" lastFinishedPulling="2026-06-02 15:47:36.41011455 +0000 UTC m=+105.684546596" observedRunningTime="2026-06-02 15:47:37.727205938 +0000 UTC m=+107.001638006" watchObservedRunningTime="2026-06-02 15:47:37.727516364 +0000 UTC m=+107.001948431" Jun 02 15:47:37.749368 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749335 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749513 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749381 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749513 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749418 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749513 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749454 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-xtksq\" (UID: \"b478c046-985d-47ab-86e3-b729b32be572\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:37.749513 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749480 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749513 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749499 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b26k8\" (UniqueName: \"kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749752 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:37.749600 2581 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found Jun 02 15:47:37.749752 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:47:37.749680 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert podName:b478c046-985d-47ab-86e3-b729b32be572 nodeName:}" failed. No retries permitted until 2026-06-02 15:47:38.249657551 +0000 UTC m=+107.524089607 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert") pod "monitoring-plugin-5768fcc9dd-xtksq" (UID: "b478c046-985d-47ab-86e3-b729b32be572") : secret "monitoring-plugin-cert" not found Jun 02 15:47:37.749865 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749840 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.749910 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.749890 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.750286 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.750241 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.750388 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.750332 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.750593 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.750576 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.750975 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.750950 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.752360 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.752340 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.752440 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.752423 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.759255 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.759236 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b26k8\" (UniqueName: \"kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8\") pod \"console-65cb97995b-rrndp\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.845669 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.845633 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:37.849290 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.849235 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-6777557fb5-tfnls"] Jun 02 15:47:37.852354 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:37.852328 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7d4f841f_616a_4743_9b42_f41fc750a150.slice/crio-ce5d24180d8245cf1c1ab49e21b8ce39da9ba668f662ee9af680d9ea016e572a WatchSource:0}: Error finding container ce5d24180d8245cf1c1ab49e21b8ce39da9ba668f662ee9af680d9ea016e572a: Status 404 returned error can't find the container with id ce5d24180d8245cf1c1ab49e21b8ce39da9ba668f662ee9af680d9ea016e572a Jun 02 15:47:37.974663 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:37.974620 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:47:37.977701 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:37.977674 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc9e330f9_f88c_4a0e_b128_4475d03ab0a1.slice/crio-bd75a076ec32cf594cdf305f384b3425c5d24f00dcf48654a9e0889119731de9 WatchSource:0}: Error finding container bd75a076ec32cf594cdf305f384b3425c5d24f00dcf48654a9e0889119731de9: Status 404 returned error can't find the container with id bd75a076ec32cf594cdf305f384b3425c5d24f00dcf48654a9e0889119731de9 Jun 02 15:47:38.253214 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.253170 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-xtksq\" (UID: \"b478c046-985d-47ab-86e3-b729b32be572\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:38.255736 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.255715 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b478c046-985d-47ab-86e3-b729b32be572-monitoring-plugin-cert\") pod \"monitoring-plugin-5768fcc9dd-xtksq\" (UID: \"b478c046-985d-47ab-86e3-b729b32be572\") " pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:38.460136 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.460094 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:38.589086 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.589045 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq"] Jun 02 15:47:38.592280 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:47:38.592235 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb478c046_985d_47ab_86e3_b729b32be572.slice/crio-d7e0cf0b04d75555b38a74500d8704e42d9de9ad2ee3630ea06f549cd007b452 WatchSource:0}: Error finding container d7e0cf0b04d75555b38a74500d8704e42d9de9ad2ee3630ea06f549cd007b452: Status 404 returned error can't find the container with id d7e0cf0b04d75555b38a74500d8704e42d9de9ad2ee3630ea06f549cd007b452 Jun 02 15:47:38.710954 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.710914 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" event={"ID":"b478c046-985d-47ab-86e3-b729b32be572","Type":"ContainerStarted","Data":"d7e0cf0b04d75555b38a74500d8704e42d9de9ad2ee3630ea06f549cd007b452"} Jun 02 15:47:38.712025 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.712006 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" event={"ID":"7d4f841f-616a-4743-9b42-f41fc750a150","Type":"ContainerStarted","Data":"ce5d24180d8245cf1c1ab49e21b8ce39da9ba668f662ee9af680d9ea016e572a"} Jun 02 15:47:38.713187 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.713163 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65cb97995b-rrndp" event={"ID":"c9e330f9-f88c-4a0e-b128-4475d03ab0a1","Type":"ContainerStarted","Data":"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d"} Jun 02 15:47:38.713290 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.713195 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65cb97995b-rrndp" event={"ID":"c9e330f9-f88c-4a0e-b128-4475d03ab0a1","Type":"ContainerStarted","Data":"bd75a076ec32cf594cdf305f384b3425c5d24f00dcf48654a9e0889119731de9"} Jun 02 15:47:38.734792 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:38.734738 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-65cb97995b-rrndp" podStartSLOduration=1.734724018 podStartE2EDuration="1.734724018s" podCreationTimestamp="2026-06-02 15:47:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:47:38.7337148 +0000 UTC m=+108.008146865" watchObservedRunningTime="2026-06-02 15:47:38.734724018 +0000 UTC m=+108.009156083" Jun 02 15:47:40.721026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:40.720987 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" event={"ID":"7d4f841f-616a-4743-9b42-f41fc750a150","Type":"ContainerStarted","Data":"e18051d2a52b04d47c66083630a21524aff2441542b3e5724bebc01e4449a8dc"} Jun 02 15:47:40.745404 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:40.745345 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" podStartSLOduration=1.044800021 podStartE2EDuration="3.745316079s" podCreationTimestamp="2026-06-02 15:47:37 +0000 UTC" firstStartedPulling="2026-06-02 15:47:37.85414152 +0000 UTC m=+107.128573562" lastFinishedPulling="2026-06-02 15:47:40.554657577 +0000 UTC m=+109.829089620" observedRunningTime="2026-06-02 15:47:40.743316416 +0000 UTC m=+110.017748480" watchObservedRunningTime="2026-06-02 15:47:40.745316079 +0000 UTC m=+110.019748145" Jun 02 15:47:41.725037 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:41.724998 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" event={"ID":"b478c046-985d-47ab-86e3-b729b32be572","Type":"ContainerStarted","Data":"edcfd042b58a0c57d2cceb391e09f16293d75f8bea598c9744a829d8f6b202c7"} Jun 02 15:47:41.725478 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:41.725289 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:41.730600 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:41.730579 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" Jun 02 15:47:41.745895 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:41.745850 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-5768fcc9dd-xtksq" podStartSLOduration=2.417199358 podStartE2EDuration="4.745837899s" podCreationTimestamp="2026-06-02 15:47:37 +0000 UTC" firstStartedPulling="2026-06-02 15:47:38.594453215 +0000 UTC m=+107.868885259" lastFinishedPulling="2026-06-02 15:47:40.923091753 +0000 UTC m=+110.197523800" observedRunningTime="2026-06-02 15:47:41.744158061 +0000 UTC m=+111.018590126" watchObservedRunningTime="2026-06-02 15:47:41.745837899 +0000 UTC m=+111.020269964" Jun 02 15:47:47.846628 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:47.846583 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:47.847128 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:47.846655 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:47.851358 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:47.851340 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:48.747005 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:48.746974 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:47:48.805700 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:48.805666 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:47:57.719318 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:57.719256 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:57.719318 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:57.719325 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:47:59.712097 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:47:59.712045 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-56d49c9f68-4tggz" podUID="a3983390-16b7-4f63-8d8e-2631ba692487" containerName="console" containerID="cri-o://57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe" gracePeriod=15 Jun 02 15:48:00.008491 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.008464 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56d49c9f68-4tggz_a3983390-16b7-4f63-8d8e-2631ba692487/console/0.log" Jun 02 15:48:00.008623 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.008535 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:48:00.018978 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.018954 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019002 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019028 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019057 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019080 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019112 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019103 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bvgk7\" (UniqueName: \"kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7\") pod \"a3983390-16b7-4f63-8d8e-2631ba692487\" (UID: \"a3983390-16b7-4f63-8d8e-2631ba692487\") " Jun 02 15:48:00.019488 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019455 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca" (OuterVolumeSpecName: "service-ca") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:00.019564 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019483 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:00.019564 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.019507 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config" (OuterVolumeSpecName: "console-config") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:00.021401 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.021377 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:00.021509 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.021459 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7" (OuterVolumeSpecName: "kube-api-access-bvgk7") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "kube-api-access-bvgk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:48:00.021549 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.021519 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a3983390-16b7-4f63-8d8e-2631ba692487" (UID: "a3983390-16b7-4f63-8d8e-2631ba692487"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:00.119944 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119911 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-service-ca\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.119944 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119943 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.119944 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119954 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-oauth-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.120175 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119963 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a3983390-16b7-4f63-8d8e-2631ba692487-console-oauth-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.120175 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119973 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a3983390-16b7-4f63-8d8e-2631ba692487-console-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.120175 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.119983 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bvgk7\" (UniqueName: \"kubernetes.io/projected/a3983390-16b7-4f63-8d8e-2631ba692487-kube-api-access-bvgk7\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:00.776620 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776596 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-56d49c9f68-4tggz_a3983390-16b7-4f63-8d8e-2631ba692487/console/0.log" Jun 02 15:48:00.776997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776634 2581 generic.go:358] "Generic (PLEG): container finished" podID="a3983390-16b7-4f63-8d8e-2631ba692487" containerID="57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe" exitCode=2 Jun 02 15:48:00.776997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776665 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d49c9f68-4tggz" event={"ID":"a3983390-16b7-4f63-8d8e-2631ba692487","Type":"ContainerDied","Data":"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe"} Jun 02 15:48:00.776997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776704 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-56d49c9f68-4tggz" event={"ID":"a3983390-16b7-4f63-8d8e-2631ba692487","Type":"ContainerDied","Data":"96148f4dc60380eca00ad3bb06a5fb32ac66199296e61e628b771adf7f650c4c"} Jun 02 15:48:00.776997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776704 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-56d49c9f68-4tggz" Jun 02 15:48:00.776997 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.776717 2581 scope.go:117] "RemoveContainer" containerID="57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe" Jun 02 15:48:00.786010 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.785993 2581 scope.go:117] "RemoveContainer" containerID="57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe" Jun 02 15:48:00.786321 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:48:00.786291 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe\": container with ID starting with 57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe not found: ID does not exist" containerID="57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe" Jun 02 15:48:00.786412 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.786330 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe"} err="failed to get container status \"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe\": rpc error: code = NotFound desc = could not find container \"57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe\": container with ID starting with 57e3b7b73dc782a1115e5ad45881f831f52314e195e058d4b3112a5e715960fe not found: ID does not exist" Jun 02 15:48:00.799614 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.799590 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:48:00.804356 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:00.804335 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-56d49c9f68-4tggz"] Jun 02 15:48:01.255881 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:01.255848 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3983390-16b7-4f63-8d8e-2631ba692487" path="/var/lib/kubelet/pods/a3983390-16b7-4f63-8d8e-2631ba692487/volumes" Jun 02 15:48:13.826016 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:13.825972 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-67dc7f488f-spld6" podUID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" containerName="console" containerID="cri-o://6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb" gracePeriod=15 Jun 02 15:48:14.085515 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.085456 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67dc7f488f-spld6_1eca3b80-b5c7-4fd3-b761-d5b00e18b17d/console/0.log" Jun 02 15:48:14.085623 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.085523 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:48:14.116081 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116048 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116085 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d94gv\" (UniqueName: \"kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116111 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116128 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116174 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116325 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116405 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116384 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert\") pod \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\" (UID: \"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d\") " Jun 02 15:48:14.116525 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116496 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:14.116583 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116551 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca" (OuterVolumeSpecName: "service-ca") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:14.116648 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116601 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config" (OuterVolumeSpecName: "console-config") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:14.116648 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116630 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-service-ca\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.116744 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.116648 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-oauth-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.117159 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.117121 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:48:14.118465 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.118438 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:14.118547 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.118506 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:48:14.118588 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.118549 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv" (OuterVolumeSpecName: "kube-api-access-d94gv") pod "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" (UID: "1eca3b80-b5c7-4fd3-b761-d5b00e18b17d"). InnerVolumeSpecName "kube-api-access-d94gv". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:48:14.217704 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.217667 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d94gv\" (UniqueName: \"kubernetes.io/projected/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-kube-api-access-d94gv\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.217704 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.217698 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-oauth-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.217704 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.217709 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.217704 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.217717 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-trusted-ca-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.217972 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.217726 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d-console-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:48:14.813387 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813356 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67dc7f488f-spld6_1eca3b80-b5c7-4fd3-b761-d5b00e18b17d/console/0.log" Jun 02 15:48:14.813560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813396 2581 generic.go:358] "Generic (PLEG): container finished" podID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" containerID="6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb" exitCode=2 Jun 02 15:48:14.813560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813457 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67dc7f488f-spld6" Jun 02 15:48:14.813560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813467 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dc7f488f-spld6" event={"ID":"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d","Type":"ContainerDied","Data":"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb"} Jun 02 15:48:14.813560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813493 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67dc7f488f-spld6" event={"ID":"1eca3b80-b5c7-4fd3-b761-d5b00e18b17d","Type":"ContainerDied","Data":"de5b8547eb79477433df99813c13748ff2085f9d8c8e3fe72a3637927fd94b39"} Jun 02 15:48:14.813560 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.813510 2581 scope.go:117] "RemoveContainer" containerID="6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb" Jun 02 15:48:14.821598 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.821580 2581 scope.go:117] "RemoveContainer" containerID="6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb" Jun 02 15:48:14.821838 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:48:14.821819 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb\": container with ID starting with 6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb not found: ID does not exist" containerID="6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb" Jun 02 15:48:14.821900 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.821846 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb"} err="failed to get container status \"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb\": rpc error: code = NotFound desc = could not find container \"6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb\": container with ID starting with 6f93cade5d5fae5ba85a8370b2e0cdd68fc6a479d267ac8a6598426aa56ef9cb not found: ID does not exist" Jun 02 15:48:14.840923 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.840899 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:48:14.849840 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:14.849819 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-67dc7f488f-spld6"] Jun 02 15:48:15.256290 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:15.256232 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" path="/var/lib/kubelet/pods/1eca3b80-b5c7-4fd3-b761-d5b00e18b17d/volumes" Jun 02 15:48:17.724439 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:17.724411 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:48:17.728284 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:17.728243 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-6777557fb5-tfnls" Jun 02 15:48:57.235187 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235150 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-965b68c99-kg9xt"] Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235404 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" containerName="console" Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235414 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" containerName="console" Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235432 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3983390-16b7-4f63-8d8e-2631ba692487" containerName="console" Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235437 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3983390-16b7-4f63-8d8e-2631ba692487" containerName="console" Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235480 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="1eca3b80-b5c7-4fd3-b761-d5b00e18b17d" containerName="console" Jun 02 15:48:57.235757 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.235489 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3983390-16b7-4f63-8d8e-2631ba692487" containerName="console" Jun 02 15:48:57.238034 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.237993 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.240716 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.240696 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\"" Jun 02 15:48:57.240840 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.240724 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-2fttb\"" Jun 02 15:48:57.240913 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.240890 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\"" Jun 02 15:48:57.241012 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.240989 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\"" Jun 02 15:48:57.241113 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.241063 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\"" Jun 02 15:48:57.241230 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.241214 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\"" Jun 02 15:48:57.250900 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.248665 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\"" Jun 02 15:48:57.258239 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.258209 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-965b68c99-kg9xt"] Jun 02 15:48:57.336023 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.335986 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-serving-certs-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336023 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336033 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-metrics-client-ca\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336234 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336080 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336305 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336258 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336362 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336331 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336362 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336348 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvqt6\" (UniqueName: \"kubernetes.io/projected/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-kube-api-access-wvqt6\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336448 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336369 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.336526 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.336508 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-federate-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.437721 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437667 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-federate-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.437721 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437721 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-serving-certs-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.437721 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437742 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-metrics-client-ca\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438019 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437771 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438019 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437797 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438019 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437836 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438019 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437861 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvqt6\" (UniqueName: \"kubernetes.io/projected/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-kube-api-access-wvqt6\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438019 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.437896 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438607 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.438569 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-serving-certs-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.438995 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.438963 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-metrics-client-ca\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.439556 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.439531 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-trusted-ca-bundle\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.440565 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.440543 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.440565 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.440558 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-secret-telemeter-client\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.440696 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.440562 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-telemeter-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.440696 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.440562 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-federate-client-tls\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.448514 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.448493 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvqt6\" (UniqueName: \"kubernetes.io/projected/ad821e36-b87c-47eb-a1c1-f7aec76d71b5-kube-api-access-wvqt6\") pod \"telemeter-client-965b68c99-kg9xt\" (UID: \"ad821e36-b87c-47eb-a1c1-f7aec76d71b5\") " pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.554672 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.554576 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" Jun 02 15:48:57.707470 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.707434 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-965b68c99-kg9xt"] Jun 02 15:48:57.713648 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:48:57.713612 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podad821e36_b87c_47eb_a1c1_f7aec76d71b5.slice/crio-256ab1dba278c45454569559ae70a79ef3bee0166a184596e99a80cacc08c39f WatchSource:0}: Error finding container 256ab1dba278c45454569559ae70a79ef3bee0166a184596e99a80cacc08c39f: Status 404 returned error can't find the container with id 256ab1dba278c45454569559ae70a79ef3bee0166a184596e99a80cacc08c39f Jun 02 15:48:57.927825 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:48:57.927792 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" event={"ID":"ad821e36-b87c-47eb-a1c1-f7aec76d71b5","Type":"ContainerStarted","Data":"256ab1dba278c45454569559ae70a79ef3bee0166a184596e99a80cacc08c39f"} Jun 02 15:49:00.938378 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:00.938343 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" event={"ID":"ad821e36-b87c-47eb-a1c1-f7aec76d71b5","Type":"ContainerStarted","Data":"9760023b12ae2cee1752805dc0d329356c92d72b4083d570a72920229966bf54"} Jun 02 15:49:01.942985 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:01.942943 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" event={"ID":"ad821e36-b87c-47eb-a1c1-f7aec76d71b5","Type":"ContainerStarted","Data":"6995f5b453705a922fd06cbab72f07dd3de944d32c276374119077c505c8fc65"} Jun 02 15:49:01.943402 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:01.942991 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" event={"ID":"ad821e36-b87c-47eb-a1c1-f7aec76d71b5","Type":"ContainerStarted","Data":"4108defc01440dffc72082332f6bc6cf964a470f4b221dc3e11904b4aef8673c"} Jun 02 15:49:01.967416 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:01.967362 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-965b68c99-kg9xt" podStartSLOduration=1.4953025119999999 podStartE2EDuration="4.967345268s" podCreationTimestamp="2026-06-02 15:48:57 +0000 UTC" firstStartedPulling="2026-06-02 15:48:57.714853636 +0000 UTC m=+186.989285678" lastFinishedPulling="2026-06-02 15:49:01.18689639 +0000 UTC m=+190.461328434" observedRunningTime="2026-06-02 15:49:01.966878895 +0000 UTC m=+191.241310962" watchObservedRunningTime="2026-06-02 15:49:01.967345268 +0000 UTC m=+191.241777332" Jun 02 15:49:02.680091 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.680052 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:02.682190 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.682167 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.699857 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.699827 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:02.784592 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784552 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9vvwg\" (UniqueName: \"kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784592 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784595 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784818 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784614 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784818 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784697 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784818 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784761 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784818 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784797 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.784818 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.784816 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.885923 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.885884 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.885923 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.885929 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.885953 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.885977 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9vvwg\" (UniqueName: \"kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.885998 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.886016 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886170 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.886054 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.886746 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.886787 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.886801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.886804 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.887133 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.887103 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.888720 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.888700 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.888809 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.888718 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.898989 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.898964 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9vvwg\" (UniqueName: \"kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg\") pod \"console-74ff554797-g7vxr\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:02.990834 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:02.990740 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:03.122801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:03.122774 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:03.125008 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:49:03.124968 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3ef2c8e4_a2ca_46a6_b840_8e25ed993b47.slice/crio-2e3e1bfd30cfd06c45385931f039e97a1c847d097c7f9ef800fbb661e2abc6e5 WatchSource:0}: Error finding container 2e3e1bfd30cfd06c45385931f039e97a1c847d097c7f9ef800fbb661e2abc6e5: Status 404 returned error can't find the container with id 2e3e1bfd30cfd06c45385931f039e97a1c847d097c7f9ef800fbb661e2abc6e5 Jun 02 15:49:03.950430 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:03.950393 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74ff554797-g7vxr" event={"ID":"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47","Type":"ContainerStarted","Data":"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be"} Jun 02 15:49:03.950430 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:03.950433 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74ff554797-g7vxr" event={"ID":"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47","Type":"ContainerStarted","Data":"2e3e1bfd30cfd06c45385931f039e97a1c847d097c7f9ef800fbb661e2abc6e5"} Jun 02 15:49:03.974676 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:03.974631 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-74ff554797-g7vxr" podStartSLOduration=1.974615054 podStartE2EDuration="1.974615054s" podCreationTimestamp="2026-06-02 15:49:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:49:03.972317388 +0000 UTC m=+193.246749453" watchObservedRunningTime="2026-06-02 15:49:03.974615054 +0000 UTC m=+193.249047422" Jun 02 15:49:12.991145 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:12.991106 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:12.991145 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:12.991150 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:12.995978 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:12.995952 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:13.982963 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:13.982938 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:14.061087 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.061048 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:49:14.197352 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.197313 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-67b464b746-pt75d"] Jun 02 15:49:14.200833 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.200807 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.220055 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.220027 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b464b746-pt75d"] Jun 02 15:49:14.285204 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285101 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-oauth-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285204 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285151 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-oauth-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285222 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-trusted-ca-bundle\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285312 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h228b\" (UniqueName: \"kubernetes.io/projected/84203b83-a969-49d6-98e8-343fae0ff315-kube-api-access-h228b\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285350 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285377 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-service-ca\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.285436 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.285407 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-console-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386248 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386196 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-oauth-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386427 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386302 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-oauth-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386427 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386357 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-trusted-ca-bundle\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386427 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386384 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h228b\" (UniqueName: \"kubernetes.io/projected/84203b83-a969-49d6-98e8-343fae0ff315-kube-api-access-h228b\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386427 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386411 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386654 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386443 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-service-ca\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386654 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386487 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-console-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.386979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.386954 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-oauth-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.387181 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.387157 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-service-ca\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.387222 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.387200 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-trusted-ca-bundle\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.387290 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.387226 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/84203b83-a969-49d6-98e8-343fae0ff315-console-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.388848 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.388827 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-oauth-config\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.389162 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.389147 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/84203b83-a969-49d6-98e8-343fae0ff315-console-serving-cert\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.398437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.398410 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h228b\" (UniqueName: \"kubernetes.io/projected/84203b83-a969-49d6-98e8-343fae0ff315-kube-api-access-h228b\") pod \"console-67b464b746-pt75d\" (UID: \"84203b83-a969-49d6-98e8-343fae0ff315\") " pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.509804 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.509763 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:14.648105 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.648082 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-67b464b746-pt75d"] Jun 02 15:49:14.650547 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:49:14.650518 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod84203b83_a969_49d6_98e8_343fae0ff315.slice/crio-546fc82e36c6ff627b62bb3a4eff011822ea4dcedb04193bb53b56f50245b7db WatchSource:0}: Error finding container 546fc82e36c6ff627b62bb3a4eff011822ea4dcedb04193bb53b56f50245b7db: Status 404 returned error can't find the container with id 546fc82e36c6ff627b62bb3a4eff011822ea4dcedb04193bb53b56f50245b7db Jun 02 15:49:14.983114 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.983073 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b464b746-pt75d" event={"ID":"84203b83-a969-49d6-98e8-343fae0ff315","Type":"ContainerStarted","Data":"5a88a2daed49ebcd03576ddc4d5882e9a231591497c29bba8ec5477eb3f34ef5"} Jun 02 15:49:14.983331 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:14.983122 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-67b464b746-pt75d" event={"ID":"84203b83-a969-49d6-98e8-343fae0ff315","Type":"ContainerStarted","Data":"546fc82e36c6ff627b62bb3a4eff011822ea4dcedb04193bb53b56f50245b7db"} Jun 02 15:49:15.004863 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:15.004818 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-67b464b746-pt75d" podStartSLOduration=1.004801638 podStartE2EDuration="1.004801638s" podCreationTimestamp="2026-06-02 15:49:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:49:15.003608064 +0000 UTC m=+204.278040139" watchObservedRunningTime="2026-06-02 15:49:15.004801638 +0000 UTC m=+204.279233704" Jun 02 15:49:24.510612 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:24.510549 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:24.510612 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:24.510610 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:24.515119 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:24.515096 2581 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:25.012708 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:25.012680 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-67b464b746-pt75d" Jun 02 15:49:25.070572 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:25.070537 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:39.082291 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.082211 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-65cb97995b-rrndp" podUID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" containerName="console" containerID="cri-o://4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d" gracePeriod=15 Jun 02 15:49:39.325766 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.325733 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65cb97995b-rrndp_c9e330f9-f88c-4a0e-b128-4475d03ab0a1/console/0.log" Jun 02 15:49:39.325899 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.325809 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:49:39.367785 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367693 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.367785 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367736 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.367785 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367771 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.368042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367804 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.368042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367827 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.368042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367852 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-b26k8\" (UniqueName: \"kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.368042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.367897 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config\") pod \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\" (UID: \"c9e330f9-f88c-4a0e-b128-4475d03ab0a1\") " Jun 02 15:49:39.368243 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.368205 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config" (OuterVolumeSpecName: "console-config") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:39.368320 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.368241 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:39.368359 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.368326 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:39.368810 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.368779 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca" (OuterVolumeSpecName: "service-ca") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:39.370223 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.370198 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:49:39.370326 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.370296 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8" (OuterVolumeSpecName: "kube-api-access-b26k8") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "kube-api-access-b26k8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:49:39.370326 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.370314 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c9e330f9-f88c-4a0e-b128-4475d03ab0a1" (UID: "c9e330f9-f88c-4a0e-b128-4475d03ab0a1"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:49:39.468421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468370 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468414 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468425 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-service-ca\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468434 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-oauth-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468421 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468444 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-b26k8\" (UniqueName: \"kubernetes.io/projected/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-kube-api-access-b26k8\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468453 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-console-oauth-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:39.468718 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:39.468463 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c9e330f9-f88c-4a0e-b128-4475d03ab0a1-trusted-ca-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:40.049607 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049578 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-65cb97995b-rrndp_c9e330f9-f88c-4a0e-b128-4475d03ab0a1/console/0.log" Jun 02 15:49:40.049805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049621 2581 generic.go:358] "Generic (PLEG): container finished" podID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" containerID="4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d" exitCode=2 Jun 02 15:49:40.049805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049661 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65cb97995b-rrndp" event={"ID":"c9e330f9-f88c-4a0e-b128-4475d03ab0a1","Type":"ContainerDied","Data":"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d"} Jun 02 15:49:40.049805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049697 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-65cb97995b-rrndp" Jun 02 15:49:40.049805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049712 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-65cb97995b-rrndp" event={"ID":"c9e330f9-f88c-4a0e-b128-4475d03ab0a1","Type":"ContainerDied","Data":"bd75a076ec32cf594cdf305f384b3425c5d24f00dcf48654a9e0889119731de9"} Jun 02 15:49:40.049805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.049732 2581 scope.go:117] "RemoveContainer" containerID="4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d" Jun 02 15:49:40.058361 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.058342 2581 scope.go:117] "RemoveContainer" containerID="4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d" Jun 02 15:49:40.058602 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:49:40.058584 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d\": container with ID starting with 4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d not found: ID does not exist" containerID="4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d" Jun 02 15:49:40.058644 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.058610 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d"} err="failed to get container status \"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d\": rpc error: code = NotFound desc = could not find container \"4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d\": container with ID starting with 4cb3eeaae855f69b1153436c545ce8590cf47f5790f61d48c28ded626893b73d not found: ID does not exist" Jun 02 15:49:40.070894 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.070870 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:49:40.077242 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:40.077220 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-65cb97995b-rrndp"] Jun 02 15:49:41.256324 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:41.256290 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" path="/var/lib/kubelet/pods/c9e330f9-f88c-4a0e-b128-4475d03ab0a1/volumes" Jun 02 15:49:50.090745 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.090689 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-74ff554797-g7vxr" podUID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" containerName="console" containerID="cri-o://4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be" gracePeriod=15 Jun 02 15:49:50.333871 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.333847 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74ff554797-g7vxr_3ef2c8e4-a2ca-46a6-b840-8e25ed993b47/console/0.log" Jun 02 15:49:50.334004 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.333918 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:50.447548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447517 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447548 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447555 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447783 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447601 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447783 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447731 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9vvwg\" (UniqueName: \"kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447886 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447798 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447886 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447841 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447987 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447886 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert\") pod \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\" (UID: \"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47\") " Jun 02 15:49:50.447987 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447912 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca" (OuterVolumeSpecName: "service-ca") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:50.447987 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.447926 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config" (OuterVolumeSpecName: "console-config") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:50.448202 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.448179 2581 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-service-ca\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.448202 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.448204 2581 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.448577 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.448254 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:50.448577 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.448328 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:49:50.449867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.449841 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:49:50.449950 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.449894 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg" (OuterVolumeSpecName: "kube-api-access-9vvwg") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "kube-api-access-9vvwg". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:49:50.449950 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.449922 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" (UID: "3ef2c8e4-a2ca-46a6-b840-8e25ed993b47"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:49:50.549056 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.549008 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9vvwg\" (UniqueName: \"kubernetes.io/projected/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-kube-api-access-9vvwg\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.549056 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.549047 2581 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.549056 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.549057 2581 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-trusted-ca-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.549056 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.549065 2581 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-oauth-serving-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:50.549383 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:50.549074 2581 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47-console-oauth-config\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:49:51.082976 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.082949 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74ff554797-g7vxr_3ef2c8e4-a2ca-46a6-b840-8e25ed993b47/console/0.log" Jun 02 15:49:51.083158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.082988 2581 generic.go:358] "Generic (PLEG): container finished" podID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" containerID="4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be" exitCode=2 Jun 02 15:49:51.083158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.083025 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74ff554797-g7vxr" event={"ID":"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47","Type":"ContainerDied","Data":"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be"} Jun 02 15:49:51.083158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.083046 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74ff554797-g7vxr" event={"ID":"3ef2c8e4-a2ca-46a6-b840-8e25ed993b47","Type":"ContainerDied","Data":"2e3e1bfd30cfd06c45385931f039e97a1c847d097c7f9ef800fbb661e2abc6e5"} Jun 02 15:49:51.083158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.083061 2581 scope.go:117] "RemoveContainer" containerID="4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be" Jun 02 15:49:51.083158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.083081 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74ff554797-g7vxr" Jun 02 15:49:51.090926 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.090908 2581 scope.go:117] "RemoveContainer" containerID="4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be" Jun 02 15:49:51.091173 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:49:51.091156 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be\": container with ID starting with 4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be not found: ID does not exist" containerID="4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be" Jun 02 15:49:51.091212 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.091182 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be"} err="failed to get container status \"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be\": rpc error: code = NotFound desc = could not find container \"4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be\": container with ID starting with 4906d239424f00d99eb0983f4142a051ce23f08a554eaaf4b2432c92e7b0a6be not found: ID does not exist" Jun 02 15:49:51.106370 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.106347 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:51.110825 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.110796 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-74ff554797-g7vxr"] Jun 02 15:49:51.256550 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:49:51.256512 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" path="/var/lib/kubelet/pods/3ef2c8e4-a2ca-46a6-b840-8e25ed993b47/volumes" Jun 02 15:50:42.679282 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679233 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd"] Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679553 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" containerName="console" Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679568 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" containerName="console" Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679578 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" containerName="console" Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679583 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" containerName="console" Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679622 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="c9e330f9-f88c-4a0e-b128-4475d03ab0a1" containerName="console" Jun 02 15:50:42.679712 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.679633 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="3ef2c8e4-a2ca-46a6-b840-8e25ed993b47" containerName="console" Jun 02 15:50:42.682232 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.682216 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.687333 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.687308 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\"" Jun 02 15:50:42.687468 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.687308 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\"" Jun 02 15:50:42.688458 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.688183 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-lrpm4\"" Jun 02 15:50:42.697078 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.697048 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd"] Jun 02 15:50:42.852022 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.851975 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.852022 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.852027 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.852259 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.852066 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zhlj8\" (UniqueName: \"kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.953494 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.953408 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.953494 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.953448 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.953685 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.953583 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zhlj8\" (UniqueName: \"kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.953771 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.953755 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.953857 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.953837 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.964128 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.964099 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zhlj8\" (UniqueName: \"kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8\") pod \"59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:42.990965 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:42.990917 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:50:43.122232 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:43.122205 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd"] Jun 02 15:50:43.124483 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:50:43.124457 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ada064d_c252_4e23_8b63_1132d80c5083.slice/crio-3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91 WatchSource:0}: Error finding container 3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91: Status 404 returned error can't find the container with id 3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91 Jun 02 15:50:43.225759 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:43.225672 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerStarted","Data":"3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91"} Jun 02 15:50:50.251073 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:50.250973 2581 generic.go:358] "Generic (PLEG): container finished" podID="5ada064d-c252-4e23-8b63-1132d80c5083" containerID="198c7a9f51fb848fd335b50f5a285d7c55e5d09a7fd1b2217ef5c94661cc2255" exitCode=0 Jun 02 15:50:50.251073 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:50.251036 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerDied","Data":"198c7a9f51fb848fd335b50f5a285d7c55e5d09a7fd1b2217ef5c94661cc2255"} Jun 02 15:50:51.161327 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:51.161297 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:50:51.161801 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:51.161780 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:50:51.169802 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:51.169778 2581 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 15:50:52.257760 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:52.257724 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerStarted","Data":"bdb761d69baf482cc2c61fefb4469c3e08b00798a75289b67dbbee39aafa53b6"} Jun 02 15:50:53.261486 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:53.261447 2581 generic.go:358] "Generic (PLEG): container finished" podID="5ada064d-c252-4e23-8b63-1132d80c5083" containerID="bdb761d69baf482cc2c61fefb4469c3e08b00798a75289b67dbbee39aafa53b6" exitCode=0 Jun 02 15:50:53.261867 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:53.261525 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerDied","Data":"bdb761d69baf482cc2c61fefb4469c3e08b00798a75289b67dbbee39aafa53b6"} Jun 02 15:50:53.262432 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:50:53.262416 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 15:51:01.286356 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:01.286314 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerStarted","Data":"6d6f74bb7afc5efa012ec4808e8c87e41728779e70598550e36839c1c44c6d64"} Jun 02 15:51:01.309903 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:01.309841 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" podStartSLOduration=1.260428819 podStartE2EDuration="19.309826601s" podCreationTimestamp="2026-06-02 15:50:42 +0000 UTC" firstStartedPulling="2026-06-02 15:50:43.126641687 +0000 UTC m=+292.401073733" lastFinishedPulling="2026-06-02 15:51:01.176039469 +0000 UTC m=+310.450471515" observedRunningTime="2026-06-02 15:51:01.308231629 +0000 UTC m=+310.582663696" watchObservedRunningTime="2026-06-02 15:51:01.309826601 +0000 UTC m=+310.584258703" Jun 02 15:51:02.291259 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:02.291214 2581 generic.go:358] "Generic (PLEG): container finished" podID="5ada064d-c252-4e23-8b63-1132d80c5083" containerID="6d6f74bb7afc5efa012ec4808e8c87e41728779e70598550e36839c1c44c6d64" exitCode=0 Jun 02 15:51:02.291638 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:02.291296 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerDied","Data":"6d6f74bb7afc5efa012ec4808e8c87e41728779e70598550e36839c1c44c6d64"} Jun 02 15:51:03.413819 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.413795 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:51:03.525592 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.525554 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-zhlj8\" (UniqueName: \"kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8\") pod \"5ada064d-c252-4e23-8b63-1132d80c5083\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " Jun 02 15:51:03.525758 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.525605 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util\") pod \"5ada064d-c252-4e23-8b63-1132d80c5083\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " Jun 02 15:51:03.525758 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.525637 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle\") pod \"5ada064d-c252-4e23-8b63-1132d80c5083\" (UID: \"5ada064d-c252-4e23-8b63-1132d80c5083\") " Jun 02 15:51:03.526173 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.526142 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle" (OuterVolumeSpecName: "bundle") pod "5ada064d-c252-4e23-8b63-1132d80c5083" (UID: "5ada064d-c252-4e23-8b63-1132d80c5083"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 15:51:03.527909 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.527877 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8" (OuterVolumeSpecName: "kube-api-access-zhlj8") pod "5ada064d-c252-4e23-8b63-1132d80c5083" (UID: "5ada064d-c252-4e23-8b63-1132d80c5083"). InnerVolumeSpecName "kube-api-access-zhlj8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:51:03.530419 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.530396 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util" (OuterVolumeSpecName: "util") pod "5ada064d-c252-4e23-8b63-1132d80c5083" (UID: "5ada064d-c252-4e23-8b63-1132d80c5083"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 15:51:03.626835 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.626727 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-zhlj8\" (UniqueName: \"kubernetes.io/projected/5ada064d-c252-4e23-8b63-1132d80c5083-kube-api-access-zhlj8\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:51:03.626835 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.626775 2581 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-util\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:51:03.626835 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:03.626785 2581 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/5ada064d-c252-4e23-8b63-1132d80c5083-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:51:04.297853 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:04.297819 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" event={"ID":"5ada064d-c252-4e23-8b63-1132d80c5083","Type":"ContainerDied","Data":"3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91"} Jun 02 15:51:04.297853 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:04.297839 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/59039e319e11338a40c6b6f1054d265f40bb50ceac6068d5c59955d29cjhqbd" Jun 02 15:51:04.297853 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:04.297851 2581 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="3bc6966a488f48df0faaf2947d98172b36b0a5617d8beab52378dd8b27166c91" Jun 02 15:51:10.425437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425396 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl"] Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425678 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="util" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425688 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="util" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425697 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="extract" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425703 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="extract" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425714 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="pull" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425719 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="pull" Jun 02 15:51:10.425790 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.425758 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="5ada064d-c252-4e23-8b63-1132d80c5083" containerName="extract" Jun 02 15:51:10.428516 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.428500 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.438494 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.438477 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"kube-root-ca.crt\"" Jun 02 15:51:10.438763 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.438750 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"custom-metrics-autoscaler-operator-dockercfg-4b766\"" Jun 02 15:51:10.439165 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.439152 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"openshift-service-ca.crt\"" Jun 02 15:51:10.440443 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.440431 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"kedaorg-certs\"" Jun 02 15:51:10.446675 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.446647 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl"] Jun 02 15:51:10.479288 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.479245 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7knpp\" (UniqueName: \"kubernetes.io/projected/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-kube-api-access-7knpp\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.479450 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.479306 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.580468 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.580424 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7knpp\" (UniqueName: \"kubernetes.io/projected/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-kube-api-access-7knpp\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.580468 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.580469 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.582978 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.582947 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/secret/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-certificates\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.593902 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.593879 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7knpp\" (UniqueName: \"kubernetes.io/projected/cda496dc-37b8-4fb6-b8eb-71ec0d1929c3-kube-api-access-7knpp\") pod \"custom-metrics-autoscaler-operator-bbf89fd5d-gckxl\" (UID: \"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3\") " pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.737838 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.737759 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:10.882743 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:10.882706 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl"] Jun 02 15:51:10.885926 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:51:10.885893 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcda496dc_37b8_4fb6_b8eb_71ec0d1929c3.slice/crio-7e40f42b03a7c0b318e915baa986554d09272e76d0a6d723ea1520b2ea70d6ce WatchSource:0}: Error finding container 7e40f42b03a7c0b318e915baa986554d09272e76d0a6d723ea1520b2ea70d6ce: Status 404 returned error can't find the container with id 7e40f42b03a7c0b318e915baa986554d09272e76d0a6d723ea1520b2ea70d6ce Jun 02 15:51:11.316979 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:11.316946 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" event={"ID":"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3","Type":"ContainerStarted","Data":"7e40f42b03a7c0b318e915baa986554d09272e76d0a6d723ea1520b2ea70d6ce"} Jun 02 15:51:15.330106 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.330069 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" event={"ID":"cda496dc-37b8-4fb6-b8eb-71ec0d1929c3","Type":"ContainerStarted","Data":"70336390b3e23b34a019349e0146425d9c3671306dde6e86b0d076eb635edeba"} Jun 02 15:51:15.330493 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.330221 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:15.359307 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.359239 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" podStartSLOduration=1.107777277 podStartE2EDuration="5.359223796s" podCreationTimestamp="2026-06-02 15:51:10 +0000 UTC" firstStartedPulling="2026-06-02 15:51:10.887606404 +0000 UTC m=+320.162038450" lastFinishedPulling="2026-06-02 15:51:15.139052925 +0000 UTC m=+324.413484969" observedRunningTime="2026-06-02 15:51:15.358305364 +0000 UTC m=+324.632737430" watchObservedRunningTime="2026-06-02 15:51:15.359223796 +0000 UTC m=+324.633655842" Jun 02 15:51:15.895256 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.895222 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-m6x6j"] Jun 02 15:51:15.898053 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.898037 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:15.908659 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.908640 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-dockercfg-t97mz\"" Jun 02 15:51:15.909035 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.909022 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-keda\"/\"keda-ocp-cabundle\"" Jun 02 15:51:15.909580 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.909567 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-operator-certs\"" Jun 02 15:51:15.931767 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:15.931739 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-m6x6j"] Jun 02 15:51:16.028774 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.028733 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/884e1ccf-373a-4f80-918e-dcc0c4e49681-cabundle0\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.028962 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.028782 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.028962 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.028853 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7ndnx\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-kube-api-access-7ndnx\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.129561 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.129528 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/884e1ccf-373a-4f80-918e-dcc0c4e49681-cabundle0\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.129733 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.129565 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.129733 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.129599 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7ndnx\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-kube-api-access-7ndnx\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.129733 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.129713 2581 secret.go:281] references non-existent secret key: ca.crt Jun 02 15:51:16.129894 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.129739 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jun 02 15:51:16.129894 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.129750 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-m6x6j: references non-existent secret key: ca.crt Jun 02 15:51:16.129894 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.129819 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates podName:884e1ccf-373a-4f80-918e-dcc0c4e49681 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:16.629798806 +0000 UTC m=+325.904230863 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates") pod "keda-operator-ffbb595cb-m6x6j" (UID: "884e1ccf-373a-4f80-918e-dcc0c4e49681") : references non-existent secret key: ca.crt Jun 02 15:51:16.130178 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.130158 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cabundle0\" (UniqueName: \"kubernetes.io/configmap/884e1ccf-373a-4f80-918e-dcc0c4e49681-cabundle0\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.162781 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.162711 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7ndnx\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-kube-api-access-7ndnx\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.369697 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.369657 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms"] Jun 02 15:51:16.372779 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.372763 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.381628 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.381609 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-metrics-apiserver-certs\"" Jun 02 15:51:16.403690 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.403662 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms"] Jun 02 15:51:16.533486 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.533446 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.533663 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.533511 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bc4782cb-cb61-446f-a7b1-3f282deffbc0-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.533663 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.533532 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jdmbt\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-kube-api-access-jdmbt\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.634316 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.634280 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.634347 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634357 2581 secret.go:281] references non-existent secret key: tls.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634377 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.634386 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bc4782cb-cb61-446f-a7b1-3f282deffbc0-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634397 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms: references non-existent secret key: tls.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.634409 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jdmbt\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-kube-api-access-jdmbt\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634458 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates podName:bc4782cb-cb61-446f-a7b1-3f282deffbc0 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:17.134443351 +0000 UTC m=+326.408875398 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates") pod "keda-metrics-apiserver-7c9f485588-pdjms" (UID: "bc4782cb-cb61-446f-a7b1-3f282deffbc0") : references non-existent secret key: tls.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634479 2581 secret.go:281] references non-existent secret key: ca.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634492 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jun 02 15:51:16.634505 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634500 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-m6x6j: references non-existent secret key: ca.crt Jun 02 15:51:16.634914 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:16.634569 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates podName:884e1ccf-373a-4f80-918e-dcc0c4e49681 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:17.634556243 +0000 UTC m=+326.908988290 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates") pod "keda-operator-ffbb595cb-m6x6j" (UID: "884e1ccf-373a-4f80-918e-dcc0c4e49681") : references non-existent secret key: ca.crt Jun 02 15:51:16.634914 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.634711 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"temp-vol\" (UniqueName: \"kubernetes.io/empty-dir/bc4782cb-cb61-446f-a7b1-3f282deffbc0-temp-vol\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.671134 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.671103 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jdmbt\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-kube-api-access-jdmbt\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:16.739415 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.739378 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-keda/keda-admission-cf49989db-2nqtv"] Jun 02 15:51:16.742564 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.742548 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.748437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.748415 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-keda\"/\"keda-admission-webhooks-certs\"" Jun 02 15:51:16.760133 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.760099 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-2nqtv"] Jun 02 15:51:16.836573 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.836474 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-certificates\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.836573 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.836512 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vwmfj\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-kube-api-access-vwmfj\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.937935 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.937899 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-certificates\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.937935 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.937937 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vwmfj\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-kube-api-access-vwmfj\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.940698 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.940672 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-certificates\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:16.967047 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:16.967012 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vwmfj\" (UniqueName: \"kubernetes.io/projected/c34f0b39-4aae-4256-96b4-afdb4993fce5-kube-api-access-vwmfj\") pod \"keda-admission-cf49989db-2nqtv\" (UID: \"c34f0b39-4aae-4256-96b4-afdb4993fce5\") " pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:17.052219 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:17.052181 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:17.142474 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:17.141810 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:17.142474 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.141990 2581 secret.go:281] references non-existent secret key: tls.crt Jun 02 15:51:17.142474 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.142007 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jun 02 15:51:17.142474 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.142029 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms: references non-existent secret key: tls.crt Jun 02 15:51:17.142474 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.142098 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates podName:bc4782cb-cb61-446f-a7b1-3f282deffbc0 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:18.142076655 +0000 UTC m=+327.416508718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates") pod "keda-metrics-apiserver-7c9f485588-pdjms" (UID: "bc4782cb-cb61-446f-a7b1-3f282deffbc0") : references non-existent secret key: tls.crt Jun 02 15:51:17.212946 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:17.212913 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-admission-cf49989db-2nqtv"] Jun 02 15:51:17.337713 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:17.337678 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-2nqtv" event={"ID":"c34f0b39-4aae-4256-96b4-afdb4993fce5","Type":"ContainerStarted","Data":"8c5fd5ef99f54fccc6e083503319842f243b5e651c1a68787ed16e4bedd28942"} Jun 02 15:51:17.646805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:17.646761 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:17.647181 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.646882 2581 secret.go:281] references non-existent secret key: ca.crt Jun 02 15:51:17.647181 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.646895 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: ca.crt Jun 02 15:51:17.647181 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.646903 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-operator-ffbb595cb-m6x6j: references non-existent secret key: ca.crt Jun 02 15:51:17.647181 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:17.646949 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates podName:884e1ccf-373a-4f80-918e-dcc0c4e49681 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:19.64693738 +0000 UTC m=+328.921369427 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates") pod "keda-operator-ffbb595cb-m6x6j" (UID: "884e1ccf-373a-4f80-918e-dcc0c4e49681") : references non-existent secret key: ca.crt Jun 02 15:51:18.151378 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:18.151339 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:18.151532 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:18.151504 2581 secret.go:281] references non-existent secret key: tls.crt Jun 02 15:51:18.151532 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:18.151526 2581 projected.go:277] Couldn't get secret payload openshift-keda/kedaorg-certs: references non-existent secret key: tls.crt Jun 02 15:51:18.151638 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:18.151550 2581 projected.go:194] Error preparing data for projected volume certificates for pod openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms: references non-existent secret key: tls.crt Jun 02 15:51:18.151638 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:51:18.151612 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates podName:bc4782cb-cb61-446f-a7b1-3f282deffbc0 nodeName:}" failed. No retries permitted until 2026-06-02 15:51:20.151592475 +0000 UTC m=+329.426024543 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "certificates" (UniqueName: "kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates") pod "keda-metrics-apiserver-7c9f485588-pdjms" (UID: "bc4782cb-cb61-446f-a7b1-3f282deffbc0") : references non-existent secret key: tls.crt Jun 02 15:51:19.348193 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.348160 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-admission-cf49989db-2nqtv" event={"ID":"c34f0b39-4aae-4256-96b4-afdb4993fce5","Type":"ContainerStarted","Data":"ac2a219803ae00ff937b5d1f75efb4672cb7e09030f0cb6ee666baa83ae8cee1"} Jun 02 15:51:19.348604 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.348294 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:19.378640 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.378594 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-admission-cf49989db-2nqtv" podStartSLOduration=2.062283071 podStartE2EDuration="3.378579065s" podCreationTimestamp="2026-06-02 15:51:16 +0000 UTC" firstStartedPulling="2026-06-02 15:51:17.218658235 +0000 UTC m=+326.493090280" lastFinishedPulling="2026-06-02 15:51:18.534954223 +0000 UTC m=+327.809386274" observedRunningTime="2026-06-02 15:51:19.377288366 +0000 UTC m=+328.651720435" watchObservedRunningTime="2026-06-02 15:51:19.378579065 +0000 UTC m=+328.653011130" Jun 02 15:51:19.663691 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.663603 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:19.666325 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.666305 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/884e1ccf-373a-4f80-918e-dcc0c4e49681-certificates\") pod \"keda-operator-ffbb595cb-m6x6j\" (UID: \"884e1ccf-373a-4f80-918e-dcc0c4e49681\") " pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:19.807185 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.807150 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:19.950536 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:19.950507 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-operator-ffbb595cb-m6x6j"] Jun 02 15:51:19.953145 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:51:19.953119 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod884e1ccf_373a_4f80_918e_dcc0c4e49681.slice/crio-80fa189e6fc6e4e3a9e8241ee06d5b50f2c9a7294faf382c233fc7a4f16f7faa WatchSource:0}: Error finding container 80fa189e6fc6e4e3a9e8241ee06d5b50f2c9a7294faf382c233fc7a4f16f7faa: Status 404 returned error can't find the container with id 80fa189e6fc6e4e3a9e8241ee06d5b50f2c9a7294faf382c233fc7a4f16f7faa Jun 02 15:51:20.167193 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:20.167158 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:20.169817 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:20.169790 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"certificates\" (UniqueName: \"kubernetes.io/projected/bc4782cb-cb61-446f-a7b1-3f282deffbc0-certificates\") pod \"keda-metrics-apiserver-7c9f485588-pdjms\" (UID: \"bc4782cb-cb61-446f-a7b1-3f282deffbc0\") " pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:20.284116 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:20.284027 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:20.353582 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:20.353541 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" event={"ID":"884e1ccf-373a-4f80-918e-dcc0c4e49681","Type":"ContainerStarted","Data":"80fa189e6fc6e4e3a9e8241ee06d5b50f2c9a7294faf382c233fc7a4f16f7faa"} Jun 02 15:51:20.417965 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:20.417938 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms"] Jun 02 15:51:20.419646 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:51:20.419621 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbc4782cb_cb61_446f_a7b1_3f282deffbc0.slice/crio-c3c771aa2b4fb093e6d7b55a57e73390c05338702f032537b4c159fe1adb34bd WatchSource:0}: Error finding container c3c771aa2b4fb093e6d7b55a57e73390c05338702f032537b4c159fe1adb34bd: Status 404 returned error can't find the container with id c3c771aa2b4fb093e6d7b55a57e73390c05338702f032537b4c159fe1adb34bd Jun 02 15:51:21.358319 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:21.358280 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" event={"ID":"bc4782cb-cb61-446f-a7b1-3f282deffbc0","Type":"ContainerStarted","Data":"c3c771aa2b4fb093e6d7b55a57e73390c05338702f032537b4c159fe1adb34bd"} Jun 02 15:51:24.368517 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.368422 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" event={"ID":"bc4782cb-cb61-446f-a7b1-3f282deffbc0","Type":"ContainerStarted","Data":"d12be05a63d106e43e5bc137fa45fc3c792f930460c48a5905b75ff07890f500"} Jun 02 15:51:24.368926 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.368542 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:24.369752 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.369731 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" event={"ID":"884e1ccf-373a-4f80-918e-dcc0c4e49681","Type":"ContainerStarted","Data":"685f7e5028f77cce39b5b68dd6097cf5d316d79689d3a0806fbecdfa7d807e47"} Jun 02 15:51:24.369854 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.369842 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:51:24.402102 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.402056 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" podStartSLOduration=4.871985784 podStartE2EDuration="8.402044494s" podCreationTimestamp="2026-06-02 15:51:16 +0000 UTC" firstStartedPulling="2026-06-02 15:51:20.420880366 +0000 UTC m=+329.695312408" lastFinishedPulling="2026-06-02 15:51:23.950939072 +0000 UTC m=+333.225371118" observedRunningTime="2026-06-02 15:51:24.400469315 +0000 UTC m=+333.674901380" watchObservedRunningTime="2026-06-02 15:51:24.402044494 +0000 UTC m=+333.676476559" Jun 02 15:51:24.429931 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:24.429887 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" podStartSLOduration=5.428726029 podStartE2EDuration="9.429874621s" podCreationTimestamp="2026-06-02 15:51:15 +0000 UTC" firstStartedPulling="2026-06-02 15:51:19.954481783 +0000 UTC m=+329.228913826" lastFinishedPulling="2026-06-02 15:51:23.955630372 +0000 UTC m=+333.230062418" observedRunningTime="2026-06-02 15:51:24.428919506 +0000 UTC m=+333.703351617" watchObservedRunningTime="2026-06-02 15:51:24.429874621 +0000 UTC m=+333.704306686" Jun 02 15:51:35.377750 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:35.377672 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-metrics-apiserver-7c9f485588-pdjms" Jun 02 15:51:36.336084 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:36.336051 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/custom-metrics-autoscaler-operator-bbf89fd5d-gckxl" Jun 02 15:51:40.356207 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:40.356176 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-admission-cf49989db-2nqtv" Jun 02 15:51:45.375369 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:51:45.375332 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-keda/keda-operator-ffbb595cb-m6x6j" Jun 02 15:52:24.744599 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.744565 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:52:24.747755 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.747739 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.750354 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.750332 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-controller-manager-dockercfg-dqzsw\"" Jun 02 15:52:24.750478 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.750366 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"openshift-service-ca.crt\"" Jun 02 15:52:24.750959 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.750943 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"kserve-webhook-server-cert\"" Jun 02 15:52:24.751042 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.751009 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve\"/\"kube-root-ca.crt\"" Jun 02 15:52:24.755659 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.755635 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-x6qqt"] Jun 02 15:52:24.758992 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.758972 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:52:24.759090 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.759066 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.761500 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.761480 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-controller-manager-dockercfg-g7bfb\"" Jun 02 15:52:24.761620 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.761608 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"llmisvc-webhook-server-cert\"" Jun 02 15:52:24.769701 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.769675 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-x6qqt"] Jun 02 15:52:24.778645 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.778622 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/seaweedfs-79f457656b-pscds"] Jun 02 15:52:24.781760 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.781739 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.785256 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.784726 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"mlpipeline-s3-artifact\"" Jun 02 15:52:24.785256 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.785003 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"default-dockercfg-8w2ng\"" Jun 02 15:52:24.795310 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.795257 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-pscds"] Jun 02 15:52:24.868711 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868676 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d04073d-b624-420f-ad4b-2f64bc1fa39e-cert\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.868925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868787 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h27cv\" (UniqueName: \"kubernetes.io/projected/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-kube-api-access-h27cv\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.868925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868821 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lqj4\" (UniqueName: \"kubernetes.io/projected/2d04073d-b624-420f-ad4b-2f64bc1fa39e-kube-api-access-6lqj4\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.868925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868854 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-data\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.868925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868875 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.868925 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.868900 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8d66d\" (UniqueName: \"kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.970026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.969981 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h27cv\" (UniqueName: \"kubernetes.io/projected/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-kube-api-access-h27cv\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.970195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970033 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lqj4\" (UniqueName: \"kubernetes.io/projected/2d04073d-b624-420f-ad4b-2f64bc1fa39e-kube-api-access-6lqj4\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.970195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970066 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-data\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.970195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970087 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.970195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970111 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8d66d\" (UniqueName: \"kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.970195 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970144 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d04073d-b624-420f-ad4b-2f64bc1fa39e-cert\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.970513 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:52:24.970235 2581 secret.go:189] Couldn't get secret kserve/kserve-webhook-server-cert: secret "kserve-webhook-server-cert" not found Jun 02 15:52:24.970513 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:52:24.970360 2581 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert podName:1e6e5251-2621-4a4f-89ec-3d31a809fdc9 nodeName:}" failed. No retries permitted until 2026-06-02 15:52:25.470337644 +0000 UTC m=+394.744769692 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert") pod "kserve-controller-manager-7b649f477f-bwnq7" (UID: "1e6e5251-2621-4a4f-89ec-3d31a809fdc9") : secret "kserve-webhook-server-cert" not found Jun 02 15:52:24.970620 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.970523 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-data\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.972945 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.972919 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/2d04073d-b624-420f-ad4b-2f64bc1fa39e-cert\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:24.979557 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.979531 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8d66d\" (UniqueName: \"kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:24.979879 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.979860 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h27cv\" (UniqueName: \"kubernetes.io/projected/dda9e362-04aa-4c5b-a453-f8ae6d729b2d-kube-api-access-h27cv\") pod \"seaweedfs-79f457656b-pscds\" (UID: \"dda9e362-04aa-4c5b-a453-f8ae6d729b2d\") " pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:24.980132 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:24.980113 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lqj4\" (UniqueName: \"kubernetes.io/projected/2d04073d-b624-420f-ad4b-2f64bc1fa39e-kube-api-access-6lqj4\") pod \"llmisvc-controller-manager-6b94ff949c-x6qqt\" (UID: \"2d04073d-b624-420f-ad4b-2f64bc1fa39e\") " pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:25.068637 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.068553 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:25.093496 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.093469 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:25.211923 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.211756 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/llmisvc-controller-manager-6b94ff949c-x6qqt"] Jun 02 15:52:25.214669 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:52:25.214643 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod2d04073d_b624_420f_ad4b_2f64bc1fa39e.slice/crio-c87002dbf4f0a93a4e650aa38db90f463167cf34121c8c6a2f6e3d403bfe902b WatchSource:0}: Error finding container c87002dbf4f0a93a4e650aa38db90f463167cf34121c8c6a2f6e3d403bfe902b: Status 404 returned error can't find the container with id c87002dbf4f0a93a4e650aa38db90f463167cf34121c8c6a2f6e3d403bfe902b Jun 02 15:52:25.229650 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.229516 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/seaweedfs-79f457656b-pscds"] Jun 02 15:52:25.231454 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:52:25.231430 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddda9e362_04aa_4c5b_a453_f8ae6d729b2d.slice/crio-2aa71824c32f9436fa318e40654c839020cbaf4b1218ea1a86772daeda001f2d WatchSource:0}: Error finding container 2aa71824c32f9436fa318e40654c839020cbaf4b1218ea1a86772daeda001f2d: Status 404 returned error can't find the container with id 2aa71824c32f9436fa318e40654c839020cbaf4b1218ea1a86772daeda001f2d Jun 02 15:52:25.475552 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.475518 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:25.477992 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.477965 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") pod \"kserve-controller-manager-7b649f477f-bwnq7\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:25.564374 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.564335 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-pscds" event={"ID":"dda9e362-04aa-4c5b-a453-f8ae6d729b2d","Type":"ContainerStarted","Data":"2aa71824c32f9436fa318e40654c839020cbaf4b1218ea1a86772daeda001f2d"} Jun 02 15:52:25.565316 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.565293 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" event={"ID":"2d04073d-b624-420f-ad4b-2f64bc1fa39e","Type":"ContainerStarted","Data":"c87002dbf4f0a93a4e650aa38db90f463167cf34121c8c6a2f6e3d403bfe902b"} Jun 02 15:52:25.658703 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.658671 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:25.893596 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:25.893461 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:52:25.968959 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:52:25.968924 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1e6e5251_2621_4a4f_89ec_3d31a809fdc9.slice/crio-3c6bef9b094af087587e8e07f1b8b0252f4fafc190ebdb0f88dde327c2d0196a WatchSource:0}: Error finding container 3c6bef9b094af087587e8e07f1b8b0252f4fafc190ebdb0f88dde327c2d0196a: Status 404 returned error can't find the container with id 3c6bef9b094af087587e8e07f1b8b0252f4fafc190ebdb0f88dde327c2d0196a Jun 02 15:52:26.570169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:26.570128 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" event={"ID":"1e6e5251-2621-4a4f-89ec-3d31a809fdc9","Type":"ContainerStarted","Data":"3c6bef9b094af087587e8e07f1b8b0252f4fafc190ebdb0f88dde327c2d0196a"} Jun 02 15:52:31.587667 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.587631 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" event={"ID":"2d04073d-b624-420f-ad4b-2f64bc1fa39e","Type":"ContainerStarted","Data":"752feb47090977cac59a9733f8c863f72a08a161bc8689ac6d03512f2b06e40b"} Jun 02 15:52:31.588122 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.587732 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:52:31.589014 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.588988 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" event={"ID":"1e6e5251-2621-4a4f-89ec-3d31a809fdc9","Type":"ContainerStarted","Data":"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9"} Jun 02 15:52:31.589150 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.589133 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:52:31.590158 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.590134 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/seaweedfs-79f457656b-pscds" event={"ID":"dda9e362-04aa-4c5b-a453-f8ae6d729b2d","Type":"ContainerStarted","Data":"77201bbbcd2f36c387d9c4bdfbb5b8d88fe36690a20e5cbb7c3c08fb5fe34e73"} Jun 02 15:52:31.590282 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.590240 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:52:31.608629 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.608589 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" podStartSLOduration=2.111124914 podStartE2EDuration="7.608578659s" podCreationTimestamp="2026-06-02 15:52:24 +0000 UTC" firstStartedPulling="2026-06-02 15:52:25.215907382 +0000 UTC m=+394.490339426" lastFinishedPulling="2026-06-02 15:52:30.713361115 +0000 UTC m=+399.987793171" observedRunningTime="2026-06-02 15:52:31.607050832 +0000 UTC m=+400.881482910" watchObservedRunningTime="2026-06-02 15:52:31.608578659 +0000 UTC m=+400.883010734" Jun 02 15:52:31.628526 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.628487 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" podStartSLOduration=2.966169783 podStartE2EDuration="7.628475249s" podCreationTimestamp="2026-06-02 15:52:24 +0000 UTC" firstStartedPulling="2026-06-02 15:52:25.970372277 +0000 UTC m=+395.244804324" lastFinishedPulling="2026-06-02 15:52:30.632677748 +0000 UTC m=+399.907109790" observedRunningTime="2026-06-02 15:52:31.627559074 +0000 UTC m=+400.901991139" watchObservedRunningTime="2026-06-02 15:52:31.628475249 +0000 UTC m=+400.902907314" Jun 02 15:52:31.647502 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:31.647455 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/seaweedfs-79f457656b-pscds" podStartSLOduration=2.117602619 podStartE2EDuration="7.647442437s" podCreationTimestamp="2026-06-02 15:52:24 +0000 UTC" firstStartedPulling="2026-06-02 15:52:25.232653676 +0000 UTC m=+394.507085719" lastFinishedPulling="2026-06-02 15:52:30.762493493 +0000 UTC m=+400.036925537" observedRunningTime="2026-06-02 15:52:31.645845579 +0000 UTC m=+400.920277645" watchObservedRunningTime="2026-06-02 15:52:31.647442437 +0000 UTC m=+400.921874502" Jun 02 15:52:37.596749 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:52:37.596715 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/seaweedfs-79f457656b-pscds" Jun 02 15:53:02.595776 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:02.595746 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/llmisvc-controller-manager-6b94ff949c-x6qqt" Jun 02 15:53:02.598775 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:02.598752 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:53:04.061906 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.061870 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:53:04.065359 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.062575 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" podUID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" containerName="manager" containerID="cri-o://dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9" gracePeriod=10 Jun 02 15:53:04.086528 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.086499 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-hsjh9"] Jun 02 15:53:04.156912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.156886 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-hsjh9"] Jun 02 15:53:04.157025 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.157008 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.192291 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.192233 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d49cd78-9671-4138-9975-c4a8deca75dd-cert\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.192447 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.192374 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9p7g8\" (UniqueName: \"kubernetes.io/projected/9d49cd78-9671-4138-9975-c4a8deca75dd-kube-api-access-9p7g8\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.293820 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.293789 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d49cd78-9671-4138-9975-c4a8deca75dd-cert\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.293975 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.293836 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9p7g8\" (UniqueName: \"kubernetes.io/projected/9d49cd78-9671-4138-9975-c4a8deca75dd-kube-api-access-9p7g8\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.296239 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.296211 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/9d49cd78-9671-4138-9975-c4a8deca75dd-cert\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.303988 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.303964 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9p7g8\" (UniqueName: \"kubernetes.io/projected/9d49cd78-9671-4138-9975-c4a8deca75dd-kube-api-access-9p7g8\") pod \"kserve-controller-manager-7b649f477f-hsjh9\" (UID: \"9d49cd78-9671-4138-9975-c4a8deca75dd\") " pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.329180 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.329158 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:53:04.394786 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.394755 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8d66d\" (UniqueName: \"kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d\") pod \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " Jun 02 15:53:04.394939 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.394820 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") pod \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\" (UID: \"1e6e5251-2621-4a4f-89ec-3d31a809fdc9\") " Jun 02 15:53:04.397022 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.396989 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert" (OuterVolumeSpecName: "cert") pod "1e6e5251-2621-4a4f-89ec-3d31a809fdc9" (UID: "1e6e5251-2621-4a4f-89ec-3d31a809fdc9"). InnerVolumeSpecName "cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:53:04.397146 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.397039 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d" (OuterVolumeSpecName: "kube-api-access-8d66d") pod "1e6e5251-2621-4a4f-89ec-3d31a809fdc9" (UID: "1e6e5251-2621-4a4f-89ec-3d31a809fdc9"). InnerVolumeSpecName "kube-api-access-8d66d". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 15:53:04.495353 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.495319 2581 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8d66d\" (UniqueName: \"kubernetes.io/projected/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-kube-api-access-8d66d\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:53:04.495353 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.495348 2581 reconciler_common.go:299] "Volume detached for volume \"cert\" (UniqueName: \"kubernetes.io/secret/1e6e5251-2621-4a4f-89ec-3d31a809fdc9-cert\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:53:04.520156 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.520125 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:04.646980 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.646917 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-hsjh9"] Jun 02 15:53:04.649174 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:53:04.649146 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9d49cd78_9671_4138_9975_c4a8deca75dd.slice/crio-764a4947136d1d9f5b74ca46200933374b78f626409bb49bf052ada64627306a WatchSource:0}: Error finding container 764a4947136d1d9f5b74ca46200933374b78f626409bb49bf052ada64627306a: Status 404 returned error can't find the container with id 764a4947136d1d9f5b74ca46200933374b78f626409bb49bf052ada64627306a Jun 02 15:53:04.693036 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.693009 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" event={"ID":"9d49cd78-9671-4138-9975-c4a8deca75dd","Type":"ContainerStarted","Data":"764a4947136d1d9f5b74ca46200933374b78f626409bb49bf052ada64627306a"} Jun 02 15:53:04.694067 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.694045 2581 generic.go:358] "Generic (PLEG): container finished" podID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" containerID="dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9" exitCode=0 Jun 02 15:53:04.694169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.694074 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" event={"ID":"1e6e5251-2621-4a4f-89ec-3d31a809fdc9","Type":"ContainerDied","Data":"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9"} Jun 02 15:53:04.694169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.694091 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" event={"ID":"1e6e5251-2621-4a4f-89ec-3d31a809fdc9","Type":"ContainerDied","Data":"3c6bef9b094af087587e8e07f1b8b0252f4fafc190ebdb0f88dde327c2d0196a"} Jun 02 15:53:04.694169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.694107 2581 scope.go:117] "RemoveContainer" containerID="dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9" Jun 02 15:53:04.694169 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.694113 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve/kserve-controller-manager-7b649f477f-bwnq7" Jun 02 15:53:04.702156 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.702139 2581 scope.go:117] "RemoveContainer" containerID="dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9" Jun 02 15:53:04.702431 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:53:04.702412 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9\": container with ID starting with dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9 not found: ID does not exist" containerID="dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9" Jun 02 15:53:04.702502 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.702443 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9"} err="failed to get container status \"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9\": rpc error: code = NotFound desc = could not find container \"dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9\": container with ID starting with dd55c12472add1974cc8711d4747e8d6b1e7c222afb83f7e5fbd84bbc1ae82b9 not found: ID does not exist" Jun 02 15:53:04.715775 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.715743 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:53:04.719950 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:04.719930 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve/kserve-controller-manager-7b649f477f-bwnq7"] Jun 02 15:53:05.257576 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:05.257471 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" path="/var/lib/kubelet/pods/1e6e5251-2621-4a4f-89ec-3d31a809fdc9/volumes" Jun 02 15:53:05.698707 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:05.698673 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" event={"ID":"9d49cd78-9671-4138-9975-c4a8deca75dd","Type":"ContainerStarted","Data":"52028e889fc76ec574dd56bf6b386b5a0055cce3ed9013be8879649b24261739"} Jun 02 15:53:05.698888 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:05.698773 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:05.716934 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:05.716862 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" podStartSLOduration=1.401682408 podStartE2EDuration="1.716848875s" podCreationTimestamp="2026-06-02 15:53:04 +0000 UTC" firstStartedPulling="2026-06-02 15:53:04.650520649 +0000 UTC m=+433.924952691" lastFinishedPulling="2026-06-02 15:53:04.965687115 +0000 UTC m=+434.240119158" observedRunningTime="2026-06-02 15:53:05.716453481 +0000 UTC m=+434.990885546" watchObservedRunningTime="2026-06-02 15:53:05.716848875 +0000 UTC m=+434.991281006" Jun 02 15:53:36.708490 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:36.708459 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/kserve-controller-manager-7b649f477f-hsjh9" Jun 02 15:53:37.595430 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.595400 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/model-serving-api-86f7b4b499-2tvn5"] Jun 02 15:53:37.595734 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.595721 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" containerName="manager" Jun 02 15:53:37.595776 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.595736 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" containerName="manager" Jun 02 15:53:37.595814 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.595784 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="1e6e5251-2621-4a4f-89ec-3d31a809fdc9" containerName="manager" Jun 02 15:53:37.598710 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.598693 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.601043 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.601019 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-tls\"" Jun 02 15:53:37.601163 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.601134 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"model-serving-api-dockercfg-2fp25\"" Jun 02 15:53:37.607425 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.607401 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-2tvn5"] Jun 02 15:53:37.610452 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.610429 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve/odh-model-controller-696fc77849-z6nbs"] Jun 02 15:53:37.613554 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.613535 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.615884 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.615861 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-webhook-cert\"" Jun 02 15:53:37.615966 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.615863 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve\"/\"odh-model-controller-dockercfg-cwwxm\"" Jun 02 15:53:37.624407 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.624385 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-z6nbs"] Jun 02 15:53:37.779796 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.779755 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-tls-certs\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.779796 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.779802 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9n58q\" (UniqueName: \"kubernetes.io/projected/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-kube-api-access-9n58q\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.780225 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.779829 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-cert\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.780225 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.779917 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lzzq\" (UniqueName: \"kubernetes.io/projected/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-kube-api-access-6lzzq\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.881364 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.881250 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-tls-certs\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.881364 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.881315 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9n58q\" (UniqueName: \"kubernetes.io/projected/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-kube-api-access-9n58q\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.881364 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.881335 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-cert\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.881583 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.881380 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lzzq\" (UniqueName: \"kubernetes.io/projected/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-kube-api-access-6lzzq\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.883815 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.883783 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-cert\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.883944 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.883853 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-tls-certs\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.889912 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.889888 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lzzq\" (UniqueName: \"kubernetes.io/projected/f0fb45a7-e90e-49fd-abab-0531eb1a7e75-kube-api-access-6lzzq\") pod \"model-serving-api-86f7b4b499-2tvn5\" (UID: \"f0fb45a7-e90e-49fd-abab-0531eb1a7e75\") " pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.890211 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.890189 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9n58q\" (UniqueName: \"kubernetes.io/projected/96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7-kube-api-access-9n58q\") pod \"odh-model-controller-696fc77849-z6nbs\" (UID: \"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7\") " pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:37.910006 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.909979 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:37.923919 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:37.923896 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:38.051150 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:38.051117 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/model-serving-api-86f7b4b499-2tvn5"] Jun 02 15:53:38.053918 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:53:38.053883 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf0fb45a7_e90e_49fd_abab_0531eb1a7e75.slice/crio-06eb3fcc89490e026cc532d31c2bbb00636547946ab9a4a7cb333604947e14ee WatchSource:0}: Error finding container 06eb3fcc89490e026cc532d31c2bbb00636547946ab9a4a7cb333604947e14ee: Status 404 returned error can't find the container with id 06eb3fcc89490e026cc532d31c2bbb00636547946ab9a4a7cb333604947e14ee Jun 02 15:53:38.077298 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:38.077251 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve/odh-model-controller-696fc77849-z6nbs"] Jun 02 15:53:38.079711 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:53:38.079680 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96bbf1c2_cd35_45f5_bdd8_e4a83f7373f7.slice/crio-ef26f51d8325e2a55d1e0ba683710241fc785764ded15bee1bcae3ce9d61b871 WatchSource:0}: Error finding container ef26f51d8325e2a55d1e0ba683710241fc785764ded15bee1bcae3ce9d61b871: Status 404 returned error can't find the container with id ef26f51d8325e2a55d1e0ba683710241fc785764ded15bee1bcae3ce9d61b871 Jun 02 15:53:38.808998 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:38.808910 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-z6nbs" event={"ID":"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7","Type":"ContainerStarted","Data":"ef26f51d8325e2a55d1e0ba683710241fc785764ded15bee1bcae3ce9d61b871"} Jun 02 15:53:38.811772 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:38.811741 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-2tvn5" event={"ID":"f0fb45a7-e90e-49fd-abab-0531eb1a7e75","Type":"ContainerStarted","Data":"06eb3fcc89490e026cc532d31c2bbb00636547946ab9a4a7cb333604947e14ee"} Jun 02 15:53:41.823670 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:41.823640 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/odh-model-controller-696fc77849-z6nbs" event={"ID":"96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7","Type":"ContainerStarted","Data":"aec2c2387ee9d1de36b0e6ccbad58cd353d41b1158290b4d5a7c5b95e32ac239"} Jun 02 15:53:41.824012 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:41.823785 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:41.846325 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:41.846285 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/odh-model-controller-696fc77849-z6nbs" podStartSLOduration=1.195713829 podStartE2EDuration="4.846252059s" podCreationTimestamp="2026-06-02 15:53:37 +0000 UTC" firstStartedPulling="2026-06-02 15:53:38.080978238 +0000 UTC m=+467.355410281" lastFinishedPulling="2026-06-02 15:53:41.731516468 +0000 UTC m=+471.005948511" observedRunningTime="2026-06-02 15:53:41.845532201 +0000 UTC m=+471.119964278" watchObservedRunningTime="2026-06-02 15:53:41.846252059 +0000 UTC m=+471.120684123" Jun 02 15:53:42.827909 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:42.827876 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve/model-serving-api-86f7b4b499-2tvn5" event={"ID":"f0fb45a7-e90e-49fd-abab-0531eb1a7e75","Type":"ContainerStarted","Data":"f67ad0f63920a6c998b8ed9d4886e09e0a32afc2c5dd1688af28fb6d9fb1ee43"} Jun 02 15:53:42.828437 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:42.828126 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:53:42.847805 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:42.847737 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve/model-serving-api-86f7b4b499-2tvn5" podStartSLOduration=2.118985568 podStartE2EDuration="5.847717528s" podCreationTimestamp="2026-06-02 15:53:37 +0000 UTC" firstStartedPulling="2026-06-02 15:53:38.055799312 +0000 UTC m=+467.330231355" lastFinishedPulling="2026-06-02 15:53:41.784531258 +0000 UTC m=+471.058963315" observedRunningTime="2026-06-02 15:53:42.84681955 +0000 UTC m=+472.121251619" watchObservedRunningTime="2026-06-02 15:53:42.847717528 +0000 UTC m=+472.122149594" Jun 02 15:53:52.830777 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:52.830747 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/odh-model-controller-696fc77849-z6nbs" Jun 02 15:53:53.839745 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:53:53.839713 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve/model-serving-api-86f7b4b499-2tvn5" Jun 02 15:55:51.189731 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:55:51.189703 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:55:51.190576 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:55:51.190558 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 15:57:39.094069 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.093978 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:57:39.097449 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.097428 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.100631 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.100609 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-l65kw\"" Jun 02 15:57:39.100744 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.100613 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jun 02 15:57:39.100804 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.100766 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-a7b79-serving-cert\"" Jun 02 15:57:39.101046 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.101027 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-a7b79-kube-rbac-proxy-sar-config\"" Jun 02 15:57:39.118606 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.118581 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:57:39.233026 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.232996 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.233213 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.233038 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.333796 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.333753 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.333999 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.333806 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.334579 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.334559 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.336383 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.336366 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls\") pod \"model-chainer-raw-a7b79-79d5b46b87-dmfxm\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.408185 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.408099 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:39.540726 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.540698 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:57:39.543253 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:57:39.543225 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60310fcf_4a9f_42bb_b713_536a434fee2e.slice/crio-d5484ae18022c6632212e0cd508e9adfe1852a297c2ac3db138e133b507507be WatchSource:0}: Error finding container d5484ae18022c6632212e0cd508e9adfe1852a297c2ac3db138e133b507507be: Status 404 returned error can't find the container with id d5484ae18022c6632212e0cd508e9adfe1852a297c2ac3db138e133b507507be Jun 02 15:57:39.545118 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.545105 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 15:57:39.599365 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:39.599329 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" event={"ID":"60310fcf-4a9f-42bb-b713-536a434fee2e","Type":"ContainerStarted","Data":"d5484ae18022c6632212e0cd508e9adfe1852a297c2ac3db138e133b507507be"} Jun 02 15:57:41.605953 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:41.605854 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" event={"ID":"60310fcf-4a9f-42bb-b713-536a434fee2e","Type":"ContainerStarted","Data":"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321"} Jun 02 15:57:41.606348 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:41.605996 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:41.630670 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:41.630620 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podStartSLOduration=0.842355622 podStartE2EDuration="2.630608008s" podCreationTimestamp="2026-06-02 15:57:39 +0000 UTC" firstStartedPulling="2026-06-02 15:57:39.545235489 +0000 UTC m=+708.819667532" lastFinishedPulling="2026-06-02 15:57:41.333487875 +0000 UTC m=+710.607919918" observedRunningTime="2026-06-02 15:57:41.628576325 +0000 UTC m=+710.903008413" watchObservedRunningTime="2026-06-02 15:57:41.630608008 +0000 UTC m=+710.905040073" Jun 02 15:57:47.616733 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:47.616700 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:57:49.306296 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:49.306241 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:57:49.306679 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:49.306466 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" containerID="cri-o://a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321" gracePeriod=30 Jun 02 15:57:52.614488 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:52.614441 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:57:57.614565 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:57:57.614524 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:58:02.614856 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:02.614816 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:58:02.615345 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:02.614958 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:58:07.614902 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:07.614858 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:58:12.615469 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:12.615424 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:58:17.614849 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:17.614806 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:58:19.330199 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:58:19.330163 2581 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60310fcf_4a9f_42bb_b713_536a434fee2e.slice/crio-conmon-a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321.scope\": RecentStats: unable to find data in memory cache]" Jun 02 15:58:19.330563 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:58:19.330176 2581 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60310fcf_4a9f_42bb_b713_536a434fee2e.slice/crio-conmon-a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321.scope\": RecentStats: unable to find data in memory cache]" Jun 02 15:58:19.330563 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:58:19.330457 2581 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod60310fcf_4a9f_42bb_b713_536a434fee2e.slice/crio-conmon-a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321.scope\": RecentStats: unable to find data in memory cache]" Jun 02 15:58:19.473359 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.473330 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:58:19.552784 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.552729 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls\") pod \"60310fcf-4a9f-42bb-b713-536a434fee2e\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " Jun 02 15:58:19.552784 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.552793 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle\") pod \"60310fcf-4a9f-42bb-b713-536a434fee2e\" (UID: \"60310fcf-4a9f-42bb-b713-536a434fee2e\") " Jun 02 15:58:19.553202 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.553170 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "60310fcf-4a9f-42bb-b713-536a434fee2e" (UID: "60310fcf-4a9f-42bb-b713-536a434fee2e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 15:58:19.555128 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.555109 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "60310fcf-4a9f-42bb-b713-536a434fee2e" (UID: "60310fcf-4a9f-42bb-b713-536a434fee2e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 15:58:19.653767 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.653663 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/60310fcf-4a9f-42bb-b713-536a434fee2e-proxy-tls\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:58:19.653767 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.653710 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/60310fcf-4a9f-42bb-b713-536a434fee2e-openshift-service-ca-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 15:58:19.731841 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.731808 2581 generic.go:358] "Generic (PLEG): container finished" podID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerID="a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321" exitCode=0 Jun 02 15:58:19.732007 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.731865 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" event={"ID":"60310fcf-4a9f-42bb-b713-536a434fee2e","Type":"ContainerDied","Data":"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321"} Jun 02 15:58:19.732007 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.731870 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" Jun 02 15:58:19.732007 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.731893 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm" event={"ID":"60310fcf-4a9f-42bb-b713-536a434fee2e","Type":"ContainerDied","Data":"d5484ae18022c6632212e0cd508e9adfe1852a297c2ac3db138e133b507507be"} Jun 02 15:58:19.732007 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.731909 2581 scope.go:117] "RemoveContainer" containerID="a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321" Jun 02 15:58:19.740206 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.740185 2581 scope.go:117] "RemoveContainer" containerID="a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321" Jun 02 15:58:19.740538 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:58:19.740520 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321\": container with ID starting with a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321 not found: ID does not exist" containerID="a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321" Jun 02 15:58:19.740604 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.740545 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321"} err="failed to get container status \"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321\": rpc error: code = NotFound desc = could not find container \"a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321\": container with ID starting with a7b6b7d783fb924be2621ba4669f208caf0c0eb66ffeab47d951d12d142e5321 not found: ID does not exist" Jun 02 15:58:19.755417 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.755385 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:58:19.761899 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:19.761874 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-a7b79-79d5b46b87-dmfxm"] Jun 02 15:58:21.257847 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:58:21.257818 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" path="/var/lib/kubelet/pods/60310fcf-4a9f-42bb-b713-536a434fee2e/volumes" Jun 02 15:59:19.724030 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.723950 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 15:59:19.724465 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.724295 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" Jun 02 15:59:19.724465 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.724309 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" Jun 02 15:59:19.724465 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.724360 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="60310fcf-4a9f-42bb-b713-536a434fee2e" containerName="model-chainer-raw-a7b79" Jun 02 15:59:19.727164 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.727148 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.730322 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.730296 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"openshift-service-ca.crt\"" Jun 02 15:59:19.730322 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.730313 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-ae9ae-kube-rbac-proxy-sar-config\"" Jun 02 15:59:19.730478 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.730439 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"model-chainer-raw-hpa-ae9ae-serving-cert\"" Jun 02 15:59:19.731235 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.731220 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kserve-ci-e2e-test\"/\"default-dockercfg-l65kw\"" Jun 02 15:59:19.736870 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.736847 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 15:59:19.765705 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.765671 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.765851 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.765742 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.866722 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.866678 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.866910 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.866743 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.867554 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.867480 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:19.869320 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:19.869299 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls\") pod \"model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:20.038323 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.038236 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:20.171566 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.171536 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 15:59:20.173425 ip-10-0-128-129 kubenswrapper[2581]: W0602 15:59:20.173398 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bdeb1f_1f19_4491_9c6e_bdf057b0f90e.slice/crio-69d4b2944a63a79c01689cf27a54ec33a6d11524545665234417b6bf16a1a200 WatchSource:0}: Error finding container 69d4b2944a63a79c01689cf27a54ec33a6d11524545665234417b6bf16a1a200: Status 404 returned error can't find the container with id 69d4b2944a63a79c01689cf27a54ec33a6d11524545665234417b6bf16a1a200 Jun 02 15:59:20.936964 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.936932 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" event={"ID":"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e","Type":"ContainerStarted","Data":"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d"} Jun 02 15:59:20.936964 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.936968 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" event={"ID":"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e","Type":"ContainerStarted","Data":"69d4b2944a63a79c01689cf27a54ec33a6d11524545665234417b6bf16a1a200"} Jun 02 15:59:20.937413 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.937004 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:20.955522 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:20.955473 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podStartSLOduration=1.955454483 podStartE2EDuration="1.955454483s" podCreationTimestamp="2026-06-02 15:59:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 15:59:20.954982988 +0000 UTC m=+810.229415056" watchObservedRunningTime="2026-06-02 15:59:20.955454483 +0000 UTC m=+810.229886551" Jun 02 15:59:26.947690 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:26.947663 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:29.787044 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:29.787012 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 15:59:29.787444 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:29.787231 2581 kuberuntime_container.go:864] "Killing container with a grace period" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" containerID="cri-o://948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d" gracePeriod=30 Jun 02 15:59:31.944984 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:31.944940 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:36.944390 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:36.944346 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:41.945063 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:41.945021 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:41.945471 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:41.945134 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 15:59:46.944290 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:46.944232 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:51.944772 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:51.944730 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:56.945147 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:56.945109 2581 prober.go:120] "Probe failed" probeType="Readiness" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 15:59:59.815113 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:59:59.815080 2581 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bdeb1f_1f19_4491_9c6e_bdf057b0f90e.slice/crio-conmon-948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d.scope\": RecentStats: unable to find data in memory cache]" Jun 02 15:59:59.815577 ip-10-0-128-129 kubenswrapper[2581]: E0602 15:59:59.815298 2581 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bdeb1f_1f19_4491_9c6e_bdf057b0f90e.slice/crio-948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0bdeb1f_1f19_4491_9c6e_bdf057b0f90e.slice/crio-conmon-948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d.scope\": RecentStats: unable to find data in memory cache]" Jun 02 15:59:59.944685 ip-10-0-128-129 kubenswrapper[2581]: I0602 15:59:59.944657 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 16:00:00.063572 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.063483 2581 generic.go:358] "Generic (PLEG): container finished" podID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerID="948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d" exitCode=0 Jun 02 16:00:00.063572 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.063556 2581 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" Jun 02 16:00:00.063758 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.063572 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" event={"ID":"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e","Type":"ContainerDied","Data":"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d"} Jun 02 16:00:00.063758 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.063613 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk" event={"ID":"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e","Type":"ContainerDied","Data":"69d4b2944a63a79c01689cf27a54ec33a6d11524545665234417b6bf16a1a200"} Jun 02 16:00:00.063758 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.063628 2581 scope.go:117] "RemoveContainer" containerID="948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d" Jun 02 16:00:00.071491 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.071472 2581 scope.go:117] "RemoveContainer" containerID="948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d" Jun 02 16:00:00.071750 ip-10-0-128-129 kubenswrapper[2581]: E0602 16:00:00.071732 2581 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d\": container with ID starting with 948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d not found: ID does not exist" containerID="948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d" Jun 02 16:00:00.071788 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.071759 2581 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d"} err="failed to get container status \"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d\": rpc error: code = NotFound desc = could not find container \"948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d\": container with ID starting with 948a502e0b8f1e3e1174662b7f978a2a7896155db819e1ab5642887c8e739b6d not found: ID does not exist" Jun 02 16:00:00.098623 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.098593 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle\") pod \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " Jun 02 16:00:00.098774 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.098632 2581 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls\") pod \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\" (UID: \"a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e\") " Jun 02 16:00:00.098941 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.098916 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle" (OuterVolumeSpecName: "openshift-service-ca-bundle") pod "a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" (UID: "a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e"). InnerVolumeSpecName "openshift-service-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 16:00:00.100796 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.100763 2581 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls" (OuterVolumeSpecName: "proxy-tls") pod "a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" (UID: "a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e"). InnerVolumeSpecName "proxy-tls". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 16:00:00.199520 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.199485 2581 reconciler_common.go:299] "Volume detached for volume \"openshift-service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-openshift-service-ca-bundle\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 16:00:00.199520 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.199516 2581 reconciler_common.go:299] "Volume detached for volume \"proxy-tls\" (UniqueName: \"kubernetes.io/secret/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e-proxy-tls\") on node \"ip-10-0-128-129.ec2.internal\" DevicePath \"\"" Jun 02 16:00:00.385310 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.385278 2581 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 16:00:00.391392 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:00.391362 2581 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kserve-ci-e2e-test/model-chainer-raw-hpa-ae9ae-6784d8f75f-hlxbk"] Jun 02 16:00:01.257230 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:01.257190 2581 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" path="/var/lib/kubelet/pods/a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e/volumes" Jun 02 16:00:51.211673 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:51.211638 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 16:00:51.213611 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:00:51.213590 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 16:05:51.233237 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:05:51.233194 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 16:05:51.235315 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:05:51.235295 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 16:08:36.503141 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:36.503094 2581 ???:1] "http: TLS handshake error from 10.0.140.246:38554: EOF" Jun 02 16:08:36.505683 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:36.505660 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-tfkkt_ff60fcd6-3fa3-4be1-b821-fdebcadc7675/global-pull-secret-syncer/0.log" Jun 02 16:08:36.638721 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:36.638692 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-tj5r4_c11ecb1c-64f7-42bc-bcd7-f09b91558d0c/konnectivity-agent/0.log" Jun 02 16:08:36.664727 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:36.664706 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-129.ec2.internal_7e0a2c8ef673f3324a54cfd46ade1a9c/haproxy/0.log" Jun 02 16:08:40.462688 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:40.462655 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-6777557fb5-tfnls_7d4f841f-616a-4743-9b42-f41fc750a150/metrics-server/0.log" Jun 02 16:08:40.494821 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:40.494792 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-5768fcc9dd-xtksq_b478c046-985d-47ab-86e3-b729b32be572/monitoring-plugin/0.log" Jun 02 16:08:40.627993 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:40.627963 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9s4q6_683ef396-e7f3-4991-99c9-c1def1c25c19/node-exporter/0.log" Jun 02 16:08:40.670286 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:40.670240 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9s4q6_683ef396-e7f3-4991-99c9-c1def1c25c19/kube-rbac-proxy/0.log" Jun 02 16:08:40.702178 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:40.702149 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-9s4q6_683ef396-e7f3-4991-99c9-c1def1c25c19/init-textfile/0.log" Jun 02 16:08:41.263494 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:41.263458 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-dzscq_1ecef6fd-3ef8-4de2-8146-f930d50696f7/prometheus-operator/0.log" Jun 02 16:08:41.297633 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:41.297605 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-658bcd8bdf-dzscq_1ecef6fd-3ef8-4de2-8146-f930d50696f7/kube-rbac-proxy/0.log" Jun 02 16:08:41.371102 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:41.371072 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-965b68c99-kg9xt_ad821e36-b87c-47eb-a1c1-f7aec76d71b5/telemeter-client/0.log" Jun 02 16:08:41.398286 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:41.398240 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-965b68c99-kg9xt_ad821e36-b87c-47eb-a1c1-f7aec76d71b5/reload/0.log" Jun 02 16:08:41.429794 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:41.429769 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-965b68c99-kg9xt_ad821e36-b87c-47eb-a1c1-f7aec76d71b5/kube-rbac-proxy/0.log" Jun 02 16:08:42.612652 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:42.612625 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-lbf7d_ce0622ed-e333-4d1a-a4ff-becd3bfa4249/networking-console-plugin/0.log" Jun 02 16:08:43.438004 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.437977 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-67b464b746-pt75d_84203b83-a969-49d6-98e8-343fae0ff315/console/0.log" Jun 02 16:08:43.470022 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.469998 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c65b8dc4b-pjz5q_2bac1ebf-7eed-458d-ac1d-05d4abca2855/download-server/0.log" Jun 02 16:08:43.605395 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.605364 2581 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm"] Jun 02 16:08:43.605776 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.605756 2581 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" Jun 02 16:08:43.605905 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.605849 2581 state_mem.go:107] "Deleted CPUSet assignment" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" Jun 02 16:08:43.605977 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.605965 2581 memory_manager.go:356] "RemoveStaleState removing state" podUID="a0bdeb1f-1f19-4491-9c6e-bdf057b0f90e" containerName="model-chainer-raw-hpa-ae9ae" Jun 02 16:08:43.608765 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.608746 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.611066 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.611050 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s989l\"/\"openshift-service-ca.crt\"" Jun 02 16:08:43.611176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.611079 2581 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-s989l\"/\"kube-root-ca.crt\"" Jun 02 16:08:43.611791 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.611776 2581 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-s989l\"/\"default-dockercfg-mv4r5\"" Jun 02 16:08:43.619087 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.619066 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm"] Jun 02 16:08:43.676408 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.676370 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-proc\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.676408 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.676408 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-lib-modules\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.676622 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.676437 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-sys\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.676622 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.676561 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lvrdv\" (UniqueName: \"kubernetes.io/projected/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-kube-api-access-lvrdv\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.676622 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.676592 2581 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-podres\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.777939 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.777831 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lvrdv\" (UniqueName: \"kubernetes.io/projected/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-kube-api-access-lvrdv\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.777939 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.777881 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-podres\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.777990 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-proc\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778012 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-podres\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778025 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-lib-modules\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778071 2581 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-sys\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778106 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-proc\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778176 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778112 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-lib-modules\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.778467 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.778176 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-sys\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.787211 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.787182 2581 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lvrdv\" (UniqueName: \"kubernetes.io/projected/4e614ffd-3e3f-40b5-bc45-1766511ae7c4-kube-api-access-lvrdv\") pod \"perf-node-gather-daemonset-jtnwm\" (UID: \"4e614ffd-3e3f-40b5-bc45-1766511ae7c4\") " pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:43.918707 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:43.918674 2581 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:44.049183 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.049156 2581 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm"] Jun 02 16:08:44.051431 ip-10-0-128-129 kubenswrapper[2581]: W0602 16:08:44.051399 2581 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod4e614ffd_3e3f_40b5_bc45_1766511ae7c4.slice/crio-9da5387c7195b82a439bab12648ae748aec53befb89394ef408d5b5c0a3ebdc9 WatchSource:0}: Error finding container 9da5387c7195b82a439bab12648ae748aec53befb89394ef408d5b5c0a3ebdc9: Status 404 returned error can't find the container with id 9da5387c7195b82a439bab12648ae748aec53befb89394ef408d5b5c0a3ebdc9 Jun 02 16:08:44.053235 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.053102 2581 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 16:08:44.558445 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.558420 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5kpc8_5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4/dns/0.log" Jun 02 16:08:44.588122 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.588097 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-5kpc8_5c7ef390-f64f-4a10-9aa6-d9fc78a36fc4/kube-rbac-proxy/0.log" Jun 02 16:08:44.763423 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.763390 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" event={"ID":"4e614ffd-3e3f-40b5-bc45-1766511ae7c4","Type":"ContainerStarted","Data":"9c05e6fe5244d25fd11d67e586b40f13c70696ce70751fed936eada21f3eb4d0"} Jun 02 16:08:44.763423 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.763425 2581 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" event={"ID":"4e614ffd-3e3f-40b5-bc45-1766511ae7c4","Type":"ContainerStarted","Data":"9da5387c7195b82a439bab12648ae748aec53befb89394ef408d5b5c0a3ebdc9"} Jun 02 16:08:44.763826 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.763530 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:44.782056 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.782013 2581 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" podStartSLOduration=1.781999544 podStartE2EDuration="1.781999544s" podCreationTimestamp="2026-06-02 16:08:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 16:08:44.780053994 +0000 UTC m=+1374.054486074" watchObservedRunningTime="2026-06-02 16:08:44.781999544 +0000 UTC m=+1374.056431609" Jun 02 16:08:44.799651 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:44.799621 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-p7f6j_2dad6529-26f1-4882-adc3-f3e371520b17/dns-node-resolver/0.log" Jun 02 16:08:45.285342 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:45.285317 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-x8t48_ebf604f8-64aa-471f-b305-bcec9fc03d7c/node-ca/0.log" Jun 02 16:08:46.361936 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:46.361904 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-fs7v5_94c86a42-2dd5-4552-8585-01550f4f9736/serve-healthcheck-canary/0.log" Jun 02 16:08:46.979780 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:46.979751 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcdrm_caa710a6-f825-44d2-96a4-d84e15cb7e52/kube-rbac-proxy/0.log" Jun 02 16:08:47.004954 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:47.004924 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcdrm_caa710a6-f825-44d2-96a4-d84e15cb7e52/exporter/0.log" Jun 02 16:08:47.050656 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:47.050626 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-rcdrm_caa710a6-f825-44d2-96a4-d84e15cb7e52/extractor/0.log" Jun 02 16:08:49.068730 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:49.068696 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_kserve-controller-manager-7b649f477f-hsjh9_9d49cd78-9671-4138-9975-c4a8deca75dd/manager/0.log" Jun 02 16:08:49.102707 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:49.102669 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_llmisvc-controller-manager-6b94ff949c-x6qqt_2d04073d-b624-420f-ad4b-2f64bc1fa39e/manager/0.log" Jun 02 16:08:49.135523 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:49.135499 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_model-serving-api-86f7b4b499-2tvn5_f0fb45a7-e90e-49fd-abab-0531eb1a7e75/server/0.log" Jun 02 16:08:49.211964 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:49.211927 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_odh-model-controller-696fc77849-z6nbs_96bbf1c2-cd35-45f5-bdd8-e4a83f7373f7/manager/0.log" Jun 02 16:08:49.284427 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:49.284396 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/kserve_seaweedfs-79f457656b-pscds_dda9e362-04aa-4c5b-a453-f8ae6d729b2d/seaweedfs/0.log" Jun 02 16:08:50.776116 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:50.776083 2581 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-s989l/perf-node-gather-daemonset-jtnwm" Jun 02 16:08:53.429851 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:53.429822 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-6gwvh_756c6be7-258a-4fa4-a371-44771f9a4529/migrator/0.log" Jun 02 16:08:53.488709 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:53.488675 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-96c6f749-6gwvh_756c6be7-258a-4fa4-a371-44771f9a4529/graceful-termination/0.log" Jun 02 16:08:55.001097 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.001069 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/kube-multus-additional-cni-plugins/0.log" Jun 02 16:08:55.026315 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.026256 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/egress-router-binary-copy/0.log" Jun 02 16:08:55.053472 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.053444 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/cni-plugins/0.log" Jun 02 16:08:55.080199 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.080170 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/bond-cni-plugin/0.log" Jun 02 16:08:55.107647 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.107611 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/routeoverride-cni/0.log" Jun 02 16:08:55.135102 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.135069 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/whereabouts-cni-bincopy/0.log" Jun 02 16:08:55.161546 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.161516 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-w896z_ea2e3887-3d5e-4848-9389-595335ede747/whereabouts-cni/0.log" Jun 02 16:08:55.379616 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.379590 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-h56zc_3e3faefc-0181-4a03-8b2c-6038fe0cb8a7/kube-multus/0.log" Jun 02 16:08:55.573391 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.573366 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rjbmw_8f9d8e73-83df-4549-b1a1-5c23188d866d/network-metrics-daemon/0.log" Jun 02 16:08:55.612111 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:55.612077 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-rjbmw_8f9d8e73-83df-4549-b1a1-5c23188d866d/kube-rbac-proxy/0.log" Jun 02 16:08:57.261888 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.261865 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-controller/0.log" Jun 02 16:08:57.299951 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.299922 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/0.log" Jun 02 16:08:57.305696 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.305680 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovn-acl-logging/1.log" Jun 02 16:08:57.346590 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.346565 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/kube-rbac-proxy-node/0.log" Jun 02 16:08:57.389623 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.389598 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 16:08:57.429120 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.429091 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/northd/0.log" Jun 02 16:08:57.472165 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.472131 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/nbdb/0.log" Jun 02 16:08:57.513766 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.513676 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/sbdb/0.log" Jun 02 16:08:57.628372 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:57.628336 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-vfcmx_da385797-83ef-450c-93ae-fbc394522e2d/ovnkube-controller/0.log" Jun 02 16:08:58.701550 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:58.701518 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-8992p_a548ced1-3bed-4d35-a80d-8aa03e106398/network-check-target-container/0.log" Jun 02 16:08:59.785743 ip-10-0-128-129 kubenswrapper[2581]: I0602 16:08:59.785706 2581 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-gwlxx_e2a2d753-3137-48b9-8f03-8781e4ccd858/iptables-alerter/0.log"