Jun 02 21:30:43.229103 ip-10-0-134-230 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory Jun 02 21:30:43.229117 ip-10-0-134-230 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory Jun 02 21:30:43.229128 ip-10-0-134-230 systemd[1]: kubelet.service: Failed with result 'resources'. Jun 02 21:30:43.229479 ip-10-0-134-230 systemd[1]: Failed to start Kubernetes Kubelet. Jun 02 21:30:53.366456 ip-10-0-134-230 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found. Jun 02 21:30:53.366476 ip-10-0-134-230 systemd[1]: kubelet.service: Failed with result 'resources'. -- Boot ab723343676d426c86ee3b263034a00c -- Jun 02 21:33:30.457477 ip-10-0-134-230 systemd[1]: Starting Kubernetes Kubelet... Jun 02 21:33:30.919380 ip-10-0-134-230 kubenswrapper[2570]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:30.919380 ip-10-0-134-230 kubenswrapper[2570]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version. Jun 02 21:33:30.919380 ip-10-0-134-230 kubenswrapper[2570]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:30.919380 ip-10-0-134-230 kubenswrapper[2570]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI. Jun 02 21:33:30.919380 ip-10-0-134-230 kubenswrapper[2570]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information. Jun 02 21:33:30.921929 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.921838 2570 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime" Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927346 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927366 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927370 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927373 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927377 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927381 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:30.927374 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927384 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927387 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927390 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927392 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927402 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927405 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927408 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927410 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927413 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927416 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927419 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927421 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927424 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927426 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927429 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927432 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927434 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927437 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927440 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927450 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:30.927633 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927453 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927456 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927460 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927465 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927468 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927471 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927473 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927476 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927479 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927481 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927484 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927487 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927489 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927492 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927495 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927497 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927500 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927503 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927505 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927508 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:30.928135 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927510 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927513 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927516 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927518 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927521 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927523 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927526 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927529 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927533 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927539 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927543 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927546 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927549 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927552 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927555 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927557 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927561 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927563 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927566 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:30.928628 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927569 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927572 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927574 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927577 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927579 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927582 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927585 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927588 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927590 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927593 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927596 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927598 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927601 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927604 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927607 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927609 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927613 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927616 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927619 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927621 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:30.929178 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.927624 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928018 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928023 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928026 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928029 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928032 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928035 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928038 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928054 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928058 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928061 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928065 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928069 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928072 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928074 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928077 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928080 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928082 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928085 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:30.929659 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928088 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928092 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928096 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928099 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928103 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928106 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928109 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928112 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928115 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928118 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928121 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928124 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928127 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928129 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928132 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928134 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928137 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928139 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928142 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:30.930129 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928145 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928147 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928150 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928153 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928156 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928158 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928161 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928163 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928166 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928168 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928171 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928173 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928177 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928179 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928182 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928184 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928187 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928189 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928191 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:30.930607 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928194 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928197 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928199 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928202 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928204 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928207 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928209 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928212 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928214 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928216 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928219 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928221 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928224 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928226 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928229 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928231 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928234 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928236 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928239 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928241 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:30.931084 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928244 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928246 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928249 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928251 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928254 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928257 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928260 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928263 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928265 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.928268 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929696 2570 flags.go:64] FLAG: --address="0.0.0.0" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929707 2570 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929714 2570 flags.go:64] FLAG: --anonymous-auth="true" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929718 2570 flags.go:64] FLAG: --application-metrics-count-limit="100" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929724 2570 flags.go:64] FLAG: --authentication-token-webhook="false" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929727 2570 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929732 2570 flags.go:64] FLAG: --authorization-mode="AlwaysAllow" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929736 2570 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929739 2570 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929742 2570 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929746 2570 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929749 2570 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki" Jun 02 21:33:30.931598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929752 2570 flags.go:64] FLAG: --cgroup-driver="cgroupfs" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929755 2570 flags.go:64] FLAG: --cgroup-root="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929758 2570 flags.go:64] FLAG: --cgroups-per-qos="true" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929761 2570 flags.go:64] FLAG: --client-ca-file="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929764 2570 flags.go:64] FLAG: --cloud-config="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929767 2570 flags.go:64] FLAG: --cloud-provider="external" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929770 2570 flags.go:64] FLAG: --cluster-dns="[]" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929774 2570 flags.go:64] FLAG: --cluster-domain="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929777 2570 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929781 2570 flags.go:64] FLAG: --config-dir="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929783 2570 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929787 2570 flags.go:64] FLAG: --container-log-max-files="5" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929791 2570 flags.go:64] FLAG: --container-log-max-size="10Mi" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929794 2570 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929797 2570 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929801 2570 flags.go:64] FLAG: --containerd-namespace="k8s.io" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929804 2570 flags.go:64] FLAG: --contention-profiling="false" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929808 2570 flags.go:64] FLAG: --cpu-cfs-quota="true" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929811 2570 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929814 2570 flags.go:64] FLAG: --cpu-manager-policy="none" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929817 2570 flags.go:64] FLAG: --cpu-manager-policy-options="" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929821 2570 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929824 2570 flags.go:64] FLAG: --enable-controller-attach-detach="true" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929827 2570 flags.go:64] FLAG: --enable-debugging-handlers="true" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929830 2570 flags.go:64] FLAG: --enable-load-reader="false" Jun 02 21:33:30.932149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929834 2570 flags.go:64] FLAG: --enable-server="true" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929837 2570 flags.go:64] FLAG: --enforce-node-allocatable="[pods]" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929841 2570 flags.go:64] FLAG: --event-burst="100" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929844 2570 flags.go:64] FLAG: --event-qps="50" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929847 2570 flags.go:64] FLAG: --event-storage-age-limit="default=0" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929850 2570 flags.go:64] FLAG: --event-storage-event-limit="default=0" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929853 2570 flags.go:64] FLAG: --eviction-hard="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929857 2570 flags.go:64] FLAG: --eviction-max-pod-grace-period="0" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929860 2570 flags.go:64] FLAG: --eviction-minimum-reclaim="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929863 2570 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929866 2570 flags.go:64] FLAG: --eviction-soft="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929869 2570 flags.go:64] FLAG: --eviction-soft-grace-period="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929871 2570 flags.go:64] FLAG: --exit-on-lock-contention="false" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929874 2570 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929877 2570 flags.go:64] FLAG: --experimental-mounter-path="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929880 2570 flags.go:64] FLAG: --fail-cgroupv1="false" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929883 2570 flags.go:64] FLAG: --fail-swap-on="true" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929886 2570 flags.go:64] FLAG: --feature-gates="" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929889 2570 flags.go:64] FLAG: --file-check-frequency="20s" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929892 2570 flags.go:64] FLAG: --global-housekeeping-interval="1m0s" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929895 2570 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929898 2570 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929901 2570 flags.go:64] FLAG: --healthz-port="10248" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929905 2570 flags.go:64] FLAG: --help="false" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929907 2570 flags.go:64] FLAG: --hostname-override="ip-10-0-134-230.ec2.internal" Jun 02 21:33:30.932773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929911 2570 flags.go:64] FLAG: --housekeeping-interval="10s" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929913 2570 flags.go:64] FLAG: --http-check-frequency="20s" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929916 2570 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929920 2570 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929923 2570 flags.go:64] FLAG: --image-gc-high-threshold="85" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929926 2570 flags.go:64] FLAG: --image-gc-low-threshold="80" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929928 2570 flags.go:64] FLAG: --image-service-endpoint="" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929931 2570 flags.go:64] FLAG: --kernel-memcg-notification="false" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929935 2570 flags.go:64] FLAG: --kube-api-burst="100" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929938 2570 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929941 2570 flags.go:64] FLAG: --kube-api-qps="50" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929944 2570 flags.go:64] FLAG: --kube-reserved="" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929946 2570 flags.go:64] FLAG: --kube-reserved-cgroup="" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929949 2570 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929952 2570 flags.go:64] FLAG: --kubelet-cgroups="" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929955 2570 flags.go:64] FLAG: --local-storage-capacity-isolation="true" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929958 2570 flags.go:64] FLAG: --lock-file="" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929961 2570 flags.go:64] FLAG: --log-cadvisor-usage="false" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929964 2570 flags.go:64] FLAG: --log-flush-frequency="5s" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929967 2570 flags.go:64] FLAG: --log-json-info-buffer-size="0" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929973 2570 flags.go:64] FLAG: --log-json-split-stream="false" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929975 2570 flags.go:64] FLAG: --log-text-info-buffer-size="0" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929978 2570 flags.go:64] FLAG: --log-text-split-stream="false" Jun 02 21:33:30.933403 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929981 2570 flags.go:64] FLAG: --logging-format="text" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929984 2570 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929987 2570 flags.go:64] FLAG: --make-iptables-util-chains="true" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929990 2570 flags.go:64] FLAG: --manifest-url="" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929993 2570 flags.go:64] FLAG: --manifest-url-header="" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.929997 2570 flags.go:64] FLAG: --max-housekeeping-interval="15s" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930001 2570 flags.go:64] FLAG: --max-open-files="1000000" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930005 2570 flags.go:64] FLAG: --max-pods="110" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930008 2570 flags.go:64] FLAG: --maximum-dead-containers="-1" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930011 2570 flags.go:64] FLAG: --maximum-dead-containers-per-container="1" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930013 2570 flags.go:64] FLAG: --memory-manager-policy="None" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930016 2570 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930019 2570 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930023 2570 flags.go:64] FLAG: --node-ip="0.0.0.0" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930026 2570 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930034 2570 flags.go:64] FLAG: --node-status-max-images="50" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930037 2570 flags.go:64] FLAG: --node-status-update-frequency="10s" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930040 2570 flags.go:64] FLAG: --oom-score-adj="-999" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930054 2570 flags.go:64] FLAG: --pod-cidr="" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930057 2570 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:f1083dc6abd9805f79797fa11c9accacc4a38caa073c94cb99097faffce05983" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930062 2570 flags.go:64] FLAG: --pod-manifest-path="" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930065 2570 flags.go:64] FLAG: --pod-max-pids="-1" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930068 2570 flags.go:64] FLAG: --pods-per-core="0" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930071 2570 flags.go:64] FLAG: --port="10250" Jun 02 21:33:30.933948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930074 2570 flags.go:64] FLAG: --protect-kernel-defaults="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930077 2570 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-06db863da2c870c90" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930081 2570 flags.go:64] FLAG: --qos-reserved="" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930083 2570 flags.go:64] FLAG: --read-only-port="10255" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930087 2570 flags.go:64] FLAG: --register-node="true" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930090 2570 flags.go:64] FLAG: --register-schedulable="true" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930092 2570 flags.go:64] FLAG: --register-with-taints="" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930096 2570 flags.go:64] FLAG: --registry-burst="10" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930099 2570 flags.go:64] FLAG: --registry-qps="5" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930102 2570 flags.go:64] FLAG: --reserved-cpus="" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930104 2570 flags.go:64] FLAG: --reserved-memory="" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930108 2570 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930111 2570 flags.go:64] FLAG: --root-dir="/var/lib/kubelet" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930114 2570 flags.go:64] FLAG: --rotate-certificates="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930120 2570 flags.go:64] FLAG: --rotate-server-certificates="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930123 2570 flags.go:64] FLAG: --runonce="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930126 2570 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930129 2570 flags.go:64] FLAG: --runtime-request-timeout="2m0s" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930133 2570 flags.go:64] FLAG: --seccomp-default="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930136 2570 flags.go:64] FLAG: --serialize-image-pulls="true" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930138 2570 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930141 2570 flags.go:64] FLAG: --storage-driver-db="cadvisor" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930145 2570 flags.go:64] FLAG: --storage-driver-host="localhost:8086" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930148 2570 flags.go:64] FLAG: --storage-driver-password="root" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930151 2570 flags.go:64] FLAG: --storage-driver-secure="false" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930154 2570 flags.go:64] FLAG: --storage-driver-table="stats" Jun 02 21:33:30.934572 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930157 2570 flags.go:64] FLAG: --storage-driver-user="root" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930160 2570 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930163 2570 flags.go:64] FLAG: --sync-frequency="1m0s" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930166 2570 flags.go:64] FLAG: --system-cgroups="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930169 2570 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930175 2570 flags.go:64] FLAG: --system-reserved-cgroup="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930177 2570 flags.go:64] FLAG: --tls-cert-file="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930180 2570 flags.go:64] FLAG: --tls-cipher-suites="[]" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930185 2570 flags.go:64] FLAG: --tls-min-version="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930188 2570 flags.go:64] FLAG: --tls-private-key-file="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930190 2570 flags.go:64] FLAG: --topology-manager-policy="none" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930193 2570 flags.go:64] FLAG: --topology-manager-policy-options="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930196 2570 flags.go:64] FLAG: --topology-manager-scope="container" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930199 2570 flags.go:64] FLAG: --v="2" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930203 2570 flags.go:64] FLAG: --version="false" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930207 2570 flags.go:64] FLAG: --vmodule="" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930211 2570 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930214 2570 flags.go:64] FLAG: --volume-stats-agg-period="1m0s" Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930308 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930311 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930316 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930319 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930322 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:30.935202 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930324 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930327 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930329 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930332 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930335 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930337 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930343 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930346 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930349 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930351 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930354 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930357 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930360 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930362 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930365 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930368 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930371 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930373 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930376 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930378 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:30.935762 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930380 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930383 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930386 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930388 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930391 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930393 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930396 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930398 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930401 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930405 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930407 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930410 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930413 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930416 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930418 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930421 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930423 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930426 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930430 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930433 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:30.936304 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930435 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930438 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930441 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930443 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930446 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930449 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930451 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930454 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930457 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930459 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930462 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930465 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930467 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930469 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930472 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930474 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930477 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930479 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930482 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930485 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:30.936799 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930487 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930491 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930494 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930496 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930499 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930501 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930504 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930506 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930509 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930511 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930515 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930519 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930524 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930528 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930531 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930534 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930537 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930541 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930544 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:30.937325 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930547 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:30.938030 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.930549 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:30.938030 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.930555 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:30.939121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.939097 2570 server.go:530] "Kubelet version" kubeletVersion="v1.33.12" Jun 02 21:33:30.939121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.939120 2570 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK="" Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939188 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939195 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939201 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939205 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939210 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939214 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939218 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939222 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939226 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939231 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939235 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939240 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939245 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939249 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939255 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939259 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939264 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939268 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939272 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:30.939268 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939276 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939281 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939285 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939289 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939293 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939297 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939302 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939307 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939311 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939315 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939320 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939326 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939331 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939335 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939340 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939344 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939348 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939352 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939356 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939360 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:30.940165 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939365 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939369 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939373 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939377 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939381 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939385 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939390 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939401 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939406 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939411 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939416 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939420 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939424 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939428 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939433 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939437 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939441 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939446 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939450 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939456 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:30.940718 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939460 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939464 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939468 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939473 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939477 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939484 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939490 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939495 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939499 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939504 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939508 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939513 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939517 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939521 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939526 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939530 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939534 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939538 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939543 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:30.941295 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939547 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939551 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939556 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939560 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939565 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939570 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939574 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939579 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.939587 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939762 2570 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939770 2570 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939774 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939780 2570 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939785 2570 feature_gate.go:328] unrecognized feature gate: ShortCertRotation Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939790 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager Jun 02 21:33:30.941923 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939794 2570 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939799 2570 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939803 2570 feature_gate.go:328] unrecognized feature gate: DNSNameResolver Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939808 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939813 2570 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939817 2570 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939822 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939826 2570 feature_gate.go:328] unrecognized feature gate: SignatureStores Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939831 2570 feature_gate.go:328] unrecognized feature gate: DualReplica Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939837 2570 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release. Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939842 2570 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939847 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939852 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939856 2570 feature_gate.go:328] unrecognized feature gate: Example2 Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939861 2570 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939865 2570 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939870 2570 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939874 2570 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939878 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939883 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPI Jun 02 21:33:30.942469 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939887 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939892 2570 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939896 2570 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939901 2570 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939906 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDC Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939910 2570 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939914 2570 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939918 2570 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939922 2570 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939927 2570 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939931 2570 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939941 2570 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939945 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939950 2570 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939954 2570 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939958 2570 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939962 2570 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939966 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939970 2570 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939974 2570 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets Jun 02 21:33:30.942957 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939978 2570 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939982 2570 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939986 2570 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939990 2570 feature_gate.go:328] unrecognized feature gate: UpgradeStatus Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.939997 2570 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release. Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940002 2570 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940007 2570 feature_gate.go:328] unrecognized feature gate: NewOLM Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940011 2570 feature_gate.go:328] unrecognized feature gate: ManagedBootImages Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940015 2570 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940020 2570 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940024 2570 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940029 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940033 2570 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940037 2570 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940060 2570 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940066 2570 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940070 2570 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940074 2570 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940078 2570 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940082 2570 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS Jun 02 21:33:30.943498 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940086 2570 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940089 2570 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940094 2570 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940098 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940102 2570 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940107 2570 feature_gate.go:328] unrecognized feature gate: OVNObservability Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940111 2570 feature_gate.go:328] unrecognized feature gate: GatewayAPIController Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940115 2570 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940119 2570 feature_gate.go:328] unrecognized feature gate: InsightsConfig Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940123 2570 feature_gate.go:328] unrecognized feature gate: Example Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940127 2570 feature_gate.go:328] unrecognized feature gate: PinnedImages Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940132 2570 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940136 2570 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940141 2570 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940145 2570 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940149 2570 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940153 2570 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940157 2570 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940161 2570 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy Jun 02 21:33:30.943985 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:30.940165 2570 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings Jun 02 21:33:30.944458 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.940172 2570 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]} Jun 02 21:33:30.944458 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.941022 2570 server.go:962] "Client rotation is on, will bootstrap in background" Jun 02 21:33:30.945692 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.945677 2570 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir" Jun 02 21:33:30.946715 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.946703 2570 server.go:1019] "Starting client certificate rotation" Jun 02 21:33:30.946819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.946803 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:33:30.946855 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.946841 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet" Jun 02 21:33:30.974745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.974728 2570 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:33:30.976750 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.976731 2570 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt" Jun 02 21:33:30.991219 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.991201 2570 log.go:25] "Validated CRI v1 runtime API" Jun 02 21:33:30.998735 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:30.998718 2570 log.go:25] "Validated CRI v1 image API" Jun 02 21:33:31.001447 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.001431 2570 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd" Jun 02 21:33:31.004463 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.004439 2570 fs.go:135] Filesystem UUIDs: map[19881efa-65bc-49c3-819e-1966ecbeb33f:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 f0d5ec81-c402-474d-91c0-cc0d5534ba62:/dev/nvme0n1p4] Jun 02 21:33:31.004529 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.004463 2570 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}] Jun 02 21:33:31.008493 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.008472 2570 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:33:31.009329 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.009216 2570 manager.go:217] Machine: {Timestamp:2026-06-02 21:33:31.008704304 +0000 UTC m=+0.435264784 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3102524 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec286b3a268c513440ce28887bafe47c SystemUUID:ec286b3a-268c-5134-40ce-28887bafe47c BootID:ab723343-676d-426c-86ee-3b263034a00c Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:8e:02:47:8e:dd Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:8e:02:47:8e:dd Speed:0 Mtu:9001} {Name:ovs-system MacAddress:da:a0:ef:4b:32:a9 Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None} Jun 02 21:33:31.009329 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.009328 2570 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available. Jun 02 21:33:31.009425 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.009402 2570 manager.go:233] Version: {KernelVersion:5.14.0-570.117.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260521-1 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:} Jun 02 21:33:31.011536 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.011511 2570 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[] Jun 02 21:33:31.011675 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.011539 2570 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-134-230.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2} Jun 02 21:33:31.012092 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.012083 2570 topology_manager.go:138] "Creating topology manager with none policy" Jun 02 21:33:31.012127 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.012095 2570 container_manager_linux.go:306] "Creating device plugin manager" Jun 02 21:33:31.012127 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.012102 2570 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:33:31.013387 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.013377 2570 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock" Jun 02 21:33:31.014656 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.014646 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:33:31.014767 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.014758 2570 server.go:1267] "Using root directory" path="/var/lib/kubelet" Jun 02 21:33:31.017460 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.017451 2570 kubelet.go:491] "Attempting to sync node with API server" Jun 02 21:33:31.017492 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.017470 2570 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests" Jun 02 21:33:31.017492 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.017483 2570 file.go:69] "Watching path" path="/etc/kubernetes/manifests" Jun 02 21:33:31.017492 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.017493 2570 kubelet.go:397] "Adding apiserver pod source" Jun 02 21:33:31.017596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.017503 2570 apiserver.go:42] "Waiting for node sync before watching apiserver pods" Jun 02 21:33:31.018531 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.018519 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:33:31.018576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.018539 2570 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking" Jun 02 21:33:31.021311 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.021294 2570 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.12-3.rhaos4.20.git64c6a00.el9" apiVersion="v1" Jun 02 21:33:31.023068 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.023034 2570 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled" Jun 02 21:33:31.025090 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025078 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025097 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025103 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025109 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025117 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025123 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025130 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi" Jun 02 21:33:31.025134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025136 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api" Jun 02 21:33:31.025308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025144 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc" Jun 02 21:33:31.025308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025150 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap" Jun 02 21:33:31.025308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025159 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected" Jun 02 21:33:31.025308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.025165 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume" Jun 02 21:33:31.026030 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.026021 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi" Jun 02 21:33:31.026075 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.026031 2570 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image" Jun 02 21:33:31.028000 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.027980 2570 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-134-230.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope Jun 02 21:33:31.028477 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.028456 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-134-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" Jun 02 21:33:31.028536 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.028467 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" Jun 02 21:33:31.029235 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.029223 2570 watchdog_linux.go:99] "Systemd watchdog is not enabled" Jun 02 21:33:31.029271 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.029263 2570 server.go:1295] "Started kubelet" Jun 02 21:33:31.029495 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.029450 2570 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10 Jun 02 21:33:31.029540 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.029515 2570 server_v1.go:47] "podresources" method="list" useActivePods=true Jun 02 21:33:31.030031 ip-10-0-134-230 systemd[1]: Started Kubernetes Kubelet. Jun 02 21:33:31.030408 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.030378 2570 server.go:180] "Starting to listen" address="0.0.0.0" port=10250 Jun 02 21:33:31.030642 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.030510 2570 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock" Jun 02 21:33:31.034910 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.034893 2570 server.go:317] "Adding debug handlers to kubelet server" Jun 02 21:33:31.037704 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.037687 2570 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving" Jun 02 21:33:31.037890 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.036247 2570 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-134-230.ec2.internal.18b5611e6eb60836 default 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-134-230.ec2.internal,UID:ip-10-0-134-230.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-134-230.ec2.internal,},FirstTimestamp:2026-06-02 21:33:31.029235766 +0000 UTC m=+0.455796245,LastTimestamp:2026-06-02 21:33:31.029235766 +0000 UTC m=+0.455796245,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-134-230.ec2.internal,}" Jun 02 21:33:31.037986 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.037964 2570 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache" Jun 02 21:33:31.038580 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.038565 2570 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer" Jun 02 21:33:31.039211 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.038913 2570 volume_manager.go:295] "The desired_state_of_world populator starts" Jun 02 21:33:31.039211 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.038926 2570 volume_manager.go:297] "Starting Kubelet Volume Manager" Jun 02 21:33:31.039211 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039008 2570 desired_state_of_world_populator.go:150] "Desired state populator starts to run" Jun 02 21:33:31.039211 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039100 2570 reconstruct.go:97] "Volume reconstruction finished" Jun 02 21:33:31.039211 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039109 2570 reconciler.go:26] "Reconciler: start to sync state" Jun 02 21:33:31.039511 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039407 2570 factory.go:55] Registering systemd factory Jun 02 21:33:31.039511 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039423 2570 factory.go:223] Registration of the systemd container factory successfully Jun 02 21:33:31.039511 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.039474 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.039638 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039626 2570 factory.go:153] Registering CRI-O factory Jun 02 21:33:31.039688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039640 2570 factory.go:223] Registration of the crio container factory successfully Jun 02 21:33:31.039736 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039690 2570 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory Jun 02 21:33:31.039736 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039706 2570 factory.go:103] Registering Raw factory Jun 02 21:33:31.039736 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.039721 2570 manager.go:1196] Started watching for new ooms in manager Jun 02 21:33:31.040150 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.040136 2570 manager.go:319] Starting recovery of all containers Jun 02 21:33:31.047160 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.047036 2570 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6" Jun 02 21:33:31.048920 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.048889 2570 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" Jun 02 21:33:31.048920 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.048898 2570 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-134-230.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms" Jun 02 21:33:31.050267 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.050250 2570 manager.go:324] Recovery completed Jun 02 21:33:31.054105 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.054093 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.054673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.054656 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jqd7" Jun 02 21:33:31.056205 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056191 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.056277 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056221 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.056277 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056236 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.056718 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056700 2570 cpu_manager.go:222] "Starting CPU manager" policy="none" Jun 02 21:33:31.056718 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056717 2570 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s" Jun 02 21:33:31.056829 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.056735 2570 state_mem.go:36] "Initialized new in-memory state store" Jun 02 21:33:31.058796 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.058785 2570 policy_none.go:49] "None policy: Start" Jun 02 21:33:31.058835 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.058800 2570 memory_manager.go:186] "Starting memorymanager" policy="None" Jun 02 21:33:31.058835 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.058810 2570 state_mem.go:35] "Initializing new in-memory state store" Jun 02 21:33:31.060262 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.060244 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-7jqd7" Jun 02 21:33:31.102644 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.102629 2570 manager.go:341] "Starting Device Plugin manager" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.102668 2570 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.102680 2570 server.go:85] "Starting device plugin registration server" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.103063 2570 eviction_manager.go:189] "Eviction manager: starting control loop" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.103083 2570 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.103159 2570 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.103255 2570 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.103265 2570 plugin_manager.go:118] "Starting Kubelet Plugin Manager" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.104236 2570 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\"" Jun 02 21:33:31.110812 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.104267 2570 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.136362 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.136342 2570 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4" Jun 02 21:33:31.136428 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.136381 2570 status_manager.go:230] "Starting to sync pod status with apiserver" Jun 02 21:33:31.136428 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.136397 2570 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started." Jun 02 21:33:31.136428 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.136404 2570 kubelet.go:2451] "Starting kubelet main sync loop" Jun 02 21:33:31.136535 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.136482 2570 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful" Jun 02 21:33:31.139586 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.139569 2570 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:31.203431 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.203337 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.204313 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.204298 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.204411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.204332 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.204411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.204347 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.204411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.204386 2570 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.210880 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.210860 2570 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.210952 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.210880 2570 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-134-230.ec2.internal\": node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.236559 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.236542 2570 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal"] Jun 02 21:33:31.236627 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.236603 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.237610 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.237589 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.237700 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.237620 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.237700 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.237635 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.238879 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.238861 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.239056 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239026 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.239146 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239075 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.239634 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239618 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.239721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239646 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.239721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239645 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.239721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239660 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.239721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239683 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.239721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.239695 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.240714 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.240699 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.240803 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.240724 2570 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach" Jun 02 21:33:31.241335 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.241312 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientMemory" Jun 02 21:33:31.241411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.241343 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasNoDiskPressure" Jun 02 21:33:31.241411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.241354 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeHasSufficientPID" Jun 02 21:33:31.262662 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.262640 2570 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-230.ec2.internal\" not found" node="ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.267161 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.267144 2570 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-134-230.ec2.internal\" not found" node="ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.285936 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.285915 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.340394 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.340370 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/51bf9710c49ae2c5fd873f7bcc330dcd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-230.ec2.internal\" (UID: \"51bf9710c49ae2c5fd873f7bcc330dcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.340495 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.340400 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.340495 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.340426 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.386486 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.386463 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.441248 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441214 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/51bf9710c49ae2c5fd873f7bcc330dcd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-230.ec2.internal\" (UID: \"51bf9710c49ae2c5fd873f7bcc330dcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.441248 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441253 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.441377 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441277 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.441377 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441334 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/51bf9710c49ae2c5fd873f7bcc330dcd-config\") pod \"kube-apiserver-proxy-ip-10-0-134-230.ec2.internal\" (UID: \"51bf9710c49ae2c5fd873f7bcc330dcd\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.441377 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441366 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.441377 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.441334 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/a0fa59b950aacfeb2c5878ddb9277dec-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal\" (UID: \"a0fa59b950aacfeb2c5878ddb9277dec\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.487502 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.487465 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.565124 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.565084 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.570749 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.570731 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.587900 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.587872 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.688413 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.688377 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.788912 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.788822 2570 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-134-230.ec2.internal\" not found" Jun 02 21:33:31.878738 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.878709 2570 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:31.939319 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.939293 2570 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.946588 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.946571 2570 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials" Jun 02 21:33:31.946703 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.946689 2570 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:33:31.946758 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.946739 2570 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received" Jun 02 21:33:31.946758 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:31.946749 2570 kubelet.go:3342] "Failed creating a mirror pod" err="Post \"https://a722803217c1e48e8beeb175b7a6e92d-1e6cea7824027be9.elb.us-east-1.amazonaws.com:6443/api/v1/namespaces/kube-system/pods\": read tcp 10.0.134.230:33204->34.231.167.31:6443: use of closed network connection" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.946842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.946774 2570 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" Jun 02 21:33:31.967306 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:31.967175 2570 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]" Jun 02 21:33:32.017976 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.017954 2570 apiserver.go:52] "Watching apiserver" Jun 02 21:33:32.024744 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.024706 2570 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66" Jun 02 21:33:32.026818 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.026796 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal","openshift-multus/multus-additional-cni-plugins-2dk4v","openshift-multus/network-metrics-daemon-vq7rr","openshift-network-diagnostics/network-check-target-scmcl","openshift-ovn-kubernetes/ovnkube-node-2ltrg","kube-system/konnectivity-agent-7w995","kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w","openshift-image-registry/node-ca-7t9s9","openshift-multus/multus-wbmzn","openshift-network-operator/iptables-alerter-dmjpb","openshift-cluster-node-tuning-operator/tuned-wf69v"] Jun 02 21:33:32.028768 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.028749 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.029627 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.029609 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.029712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.029671 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:32.029767 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.029703 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:32.029818 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.029786 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:32.030734 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.030716 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.031308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.031288 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.031435 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.031336 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\"" Jun 02 21:33:32.031435 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.031367 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\"" Jun 02 21:33:32.031611 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.031598 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.031847 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.031829 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.032238 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.032219 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\"" Jun 02 21:33:32.032362 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.032347 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-x5bbk\"" Jun 02 21:33:32.032887 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.032872 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\"" Jun 02 21:33:32.032966 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.032956 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\"" Jun 02 21:33:32.033070 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.033038 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.034188 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.034111 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.035244 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.034711 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\"" Jun 02 21:33:32.035244 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.034896 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-6gcvk\"" Jun 02 21:33:32.035244 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.034948 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\"" Jun 02 21:33:32.035702 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.035680 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\"" Jun 02 21:33:32.035795 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.035723 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\"" Jun 02 21:33:32.035795 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.035748 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\"" Jun 02 21:33:32.036716 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.036695 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.036812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.036799 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.036909 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.036889 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.036982 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.036958 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-jbfwk\"" Jun 02 21:33:32.037034 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.036958 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.037208 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.037189 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-nn8m7\"" Jun 02 21:33:32.037762 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.037745 2570 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving" Jun 02 21:33:32.038073 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.038035 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.039069 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.038996 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\"" Jun 02 21:33:32.039069 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.039019 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.039204 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.039102 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-dbd9d\"" Jun 02 21:33:32.039204 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.039140 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.039311 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.039289 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.040278 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.040262 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.040615 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.040597 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-8922f\"" Jun 02 21:33:32.040693 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.040625 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\"" Jun 02 21:33:32.041557 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.041539 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.041718 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.041703 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\"" Jun 02 21:33:32.041805 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.041791 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.041874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.041792 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-kfnf4\"" Jun 02 21:33:32.042139 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.042124 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:33:32.042217 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.042126 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-54nlb\"" Jun 02 21:33:32.042366 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.042351 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\"" Jun 02 21:33:32.043513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043495 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-os-release\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.043603 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043527 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-netns\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.043603 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043550 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-systemd-units\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043603 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043576 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043600 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-multus\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043626 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-conf-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043652 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x46zz\" (UniqueName: \"kubernetes.io/projected/291f68dc-768f-463f-8177-f9b25147390e-kube-api-access-x46zz\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043676 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-kubelet\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043699 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-slash\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-var-lib-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-ovn\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043786 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-bin\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:32.043874 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043853 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-device-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043887 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-bin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9r9t\" (UniqueName: \"kubernetes.io/projected/76b9dfcd-868b-46a7-8e10-4b624786b63e-kube-api-access-p9r9t\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043939 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.043964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-systemd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044003 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-node-log\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044066 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r989\" (UniqueName: \"kubernetes.io/projected/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kube-api-access-5r989\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044094 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-cnibin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044112 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044132 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-netd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6tng\" (UniqueName: \"kubernetes.io/projected/c682cb62-1356-4ee4-ad9a-f7a368bdba49-kube-api-access-p6tng\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044176 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9afa5d28-b5ca-4780-8f04-89166c9b1054-serviceca\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044196 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-log-socket\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044212 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044229 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0b314d-538c-4868-a86d-72964ae80f4a-host-slash\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-system-cni-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.044379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044294 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044322 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovn-node-metrics-cert\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044344 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9afa5d28-b5ca-4780-8f04-89166c9b1054-host\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044381 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-socket-dir-parent\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044429 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-etc-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044459 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10581f86-1587-4d24-97fc-7630c1309315-agent-certs\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044496 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-sys-fs\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044532 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-cni-binary-copy\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044554 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-etc-kubernetes\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044595 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0b314d-538c-4868-a86d-72964ae80f4a-iptables-alerter-script\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044626 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-script-lib\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044650 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044673 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-os-release\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044702 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-daemon-config\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044725 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-env-overrides\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044749 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q266w\" (UniqueName: \"kubernetes.io/projected/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-kube-api-access-q266w\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.045121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044784 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9fp54\" (UniqueName: \"kubernetes.io/projected/be0b314d-538c-4868-a86d-72964ae80f4a-kube-api-access-9fp54\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044814 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bglvj\" (UniqueName: \"kubernetes.io/projected/9afa5d28-b5ca-4780-8f04-89166c9b1054-kube-api-access-bglvj\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044884 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044920 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-kubelet\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-hostroot\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044960 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.044984 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-config\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045009 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-socket-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045035 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-system-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045095 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045120 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-cnibin\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045144 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10581f86-1587-4d24-97fc-7630c1309315-konnectivity-ca\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045167 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-multus-certs\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045190 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-registration-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.045760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-k8s-cni-cncf-io\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.046241 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.045242 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-netns\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.052074 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.052039 2570 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162" Jun 02 21:33:32.060590 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.060567 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod51bf9710c49ae2c5fd873f7bcc330dcd.slice/crio-b4c714f30fe0e1e5d75f26b23ee8fa4f40bf72b3957375ce3b626e97d95539bd WatchSource:0}: Error finding container b4c714f30fe0e1e5d75f26b23ee8fa4f40bf72b3957375ce3b626e97d95539bd: Status 404 returned error can't find the container with id b4c714f30fe0e1e5d75f26b23ee8fa4f40bf72b3957375ce3b626e97d95539bd Jun 02 21:33:32.060776 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.060753 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda0fa59b950aacfeb2c5878ddb9277dec.slice/crio-a83ddad2152fc48b9db6c6d5a28d8cad0886dfa046f758f4e82f79197661d8f6 WatchSource:0}: Error finding container a83ddad2152fc48b9db6c6d5a28d8cad0886dfa046f758f4e82f79197661d8f6: Status 404 returned error can't find the container with id a83ddad2152fc48b9db6c6d5a28d8cad0886dfa046f758f4e82f79197661d8f6 Jun 02 21:33:32.061845 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.061821 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-06-01 21:28:31 +0000 UTC" deadline="2027-10-31 00:11:51.173843288 +0000 UTC" Jun 02 21:33:32.061845 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.061840 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12362h38m19.11200486s" Jun 02 21:33:32.063892 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.063878 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:33:32.079513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.079496 2570 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-w2hs9" Jun 02 21:33:32.087410 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.087390 2570 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-w2hs9" Jun 02 21:33:32.139139 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.139089 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" event={"ID":"a0fa59b950aacfeb2c5878ddb9277dec","Type":"ContainerStarted","Data":"a83ddad2152fc48b9db6c6d5a28d8cad0886dfa046f758f4e82f79197661d8f6"} Jun 02 21:33:32.139731 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.139707 2570 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world" Jun 02 21:33:32.140004 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.139985 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" event={"ID":"51bf9710c49ae2c5fd873f7bcc330dcd","Type":"ContainerStarted","Data":"b4c714f30fe0e1e5d75f26b23ee8fa4f40bf72b3957375ce3b626e97d95539bd"} Jun 02 21:33:32.145484 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145458 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-tmp\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.145597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145495 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-multus\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145520 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-conf-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145543 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x46zz\" (UniqueName: \"kubernetes.io/projected/291f68dc-768f-463f-8177-f9b25147390e-kube-api-access-x46zz\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.145597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145576 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-conf-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145576 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-multus\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145568 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-kubelet\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145615 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-kubelet\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145631 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-slash\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145652 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-var-lib-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145669 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-ovn\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145684 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-bin\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145700 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145721 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-device-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145727 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-var-lib-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145738 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-bin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145759 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9r9t\" (UniqueName: \"kubernetes.io/projected/76b9dfcd-868b-46a7-8e10-4b624786b63e-kube-api-access-p9r9t\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145768 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-slash\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145786 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-device-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145772 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-bin\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145734 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-ovn\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145777 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.145864 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145810 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-cni-bin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145840 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-systemd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145948 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-node-log\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145960 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-systemd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.145985 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5r989\" (UniqueName: \"kubernetes.io/projected/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kube-api-access-5r989\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146013 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-cnibin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146072 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146078 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-node-log\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146096 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146103 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-netd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146085 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-cnibin\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146129 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p6tng\" (UniqueName: \"kubernetes.io/projected/c682cb62-1356-4ee4-ad9a-f7a368bdba49-kube-api-access-p6tng\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146158 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-cni-netd\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146157 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9afa5d28-b5ca-4780-8f04-89166c9b1054-serviceca\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146199 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-host\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146225 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-log-socket\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146257 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.146626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146283 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0b314d-538c-4868-a86d-72964ae80f4a-host-slash\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146286 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146309 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-system-cni-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146288 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-log-socket\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146334 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146341 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kubelet-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146359 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovn-node-metrics-cert\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146363 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-system-cni-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146355 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/be0b314d-538c-4868-a86d-72964ae80f4a-host-slash\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146457 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9afa5d28-b5ca-4780-8f04-89166c9b1054-host\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146478 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysconfig\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146482 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-tuning-conf-dir\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146496 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-socket-dir-parent\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146513 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/9afa5d28-b5ca-4780-8f04-89166c9b1054-host\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146527 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-etc-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146536 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-socket-dir-parent\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146554 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10581f86-1587-4d24-97fc-7630c1309315-agent-certs\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.147344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146570 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-etc-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146530 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/9afa5d28-b5ca-4780-8f04-89166c9b1054-serviceca\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146571 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-lib-modules\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146607 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-sys-fs\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146623 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-cni-binary-copy\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146639 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-etc-kubernetes\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146654 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0b314d-538c-4868-a86d-72964ae80f4a-iptables-alerter-script\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146676 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-script-lib\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146681 2570 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146693 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-sys-fs\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146784 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146802 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-etc-kubernetes\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146813 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-os-release\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146840 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-daemon-config\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146863 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-env-overrides\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146889 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-conf\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146913 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8cfqr\" (UniqueName: \"kubernetes.io/projected/472bb351-9e91-46f6-bcd5-fb53b06a2aad-kube-api-access-8cfqr\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148135 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146941 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q266w\" (UniqueName: \"kubernetes.io/projected/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-kube-api-access-q266w\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146971 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9fp54\" (UniqueName: \"kubernetes.io/projected/be0b314d-538c-4868-a86d-72964ae80f4a-kube-api-access-9fp54\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.146996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-run\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147020 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147097 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bglvj\" (UniqueName: \"kubernetes.io/projected/9afa5d28-b5ca-4780-8f04-89166c9b1054-kube-api-access-bglvj\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147126 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147154 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147179 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-kubelet\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147204 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-hostroot\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147211 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/be0b314d-538c-4868-a86d-72964ae80f4a-iptables-alerter-script\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147230 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147243 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-script-lib\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147256 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-config\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147305 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-modprobe-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147329 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147356 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-socket-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147389 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-system-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.148932 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-cni-binary-copy\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147432 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147464 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-cnibin\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147479 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-os-release\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147492 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10581f86-1587-4d24-97fc-7630c1309315-konnectivity-ca\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147521 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-systemd\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147529 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-etc-selinux\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147546 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-tuned\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147573 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-multus-certs\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147598 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-registration-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147600 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-env-overrides\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147626 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-k8s-cni-cncf-io\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147662 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-netns\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147729 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-k8s-cni-cncf-io\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147743 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-cnibin\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147761 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-os-release\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147784 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-kubernetes\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.149673 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147807 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-sys\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147829 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-var-lib-kubelet\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147815 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-run-netns\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147855 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-netns\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147875 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147878 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-systemd-units\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147909 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-systemd-units\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147916 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147973 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-daemon-config\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147982 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-multus-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/c682cb62-1356-4ee4-ad9a-f7a368bdba49-run-openvswitch\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.147783 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-var-lib-kubelet\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148073 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-system-cni-dir\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148076 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-hostroot\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148109 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-socket-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148110 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148112 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-multus-certs\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.148141 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:32.150165 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148154 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-registration-dir\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.148209 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:33:32.648190481 +0000 UTC m=+2.074750963 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148207 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/291f68dc-768f-463f-8177-f9b25147390e-os-release\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148245 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/76b9dfcd-868b-46a7-8e10-4b624786b63e-host-run-netns\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148396 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/10581f86-1587-4d24-97fc-7630c1309315-konnectivity-ca\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148523 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/291f68dc-768f-463f-8177-f9b25147390e-cni-binary-copy\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.148532 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovnkube-config\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.149873 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/c682cb62-1356-4ee4-ad9a-f7a368bdba49-ovn-node-metrics-cert\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.150622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.149878 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/10581f86-1587-4d24-97fc-7630c1309315-agent-certs\") pod \"konnectivity-agent-7w995\" (UID: \"10581f86-1587-4d24-97fc-7630c1309315\") " pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.152857 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.152836 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:32.152971 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.152862 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:32.152971 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.152876 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:32.152971 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.152945 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:33:32.652928141 +0000 UTC m=+2.079488649 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:32.153965 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.153912 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x46zz\" (UniqueName: \"kubernetes.io/projected/291f68dc-768f-463f-8177-f9b25147390e-kube-api-access-x46zz\") pod \"multus-additional-cni-plugins-2dk4v\" (UID: \"291f68dc-768f-463f-8177-f9b25147390e\") " pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.154215 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.154195 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9r9t\" (UniqueName: \"kubernetes.io/projected/76b9dfcd-868b-46a7-8e10-4b624786b63e-kube-api-access-p9r9t\") pod \"multus-wbmzn\" (UID: \"76b9dfcd-868b-46a7-8e10-4b624786b63e\") " pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.154382 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.154363 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6tng\" (UniqueName: \"kubernetes.io/projected/c682cb62-1356-4ee4-ad9a-f7a368bdba49-kube-api-access-p6tng\") pod \"ovnkube-node-2ltrg\" (UID: \"c682cb62-1356-4ee4-ad9a-f7a368bdba49\") " pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.154667 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.154653 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r989\" (UniqueName: \"kubernetes.io/projected/f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8-kube-api-access-5r989\") pod \"aws-ebs-csi-driver-node-ksk6w\" (UID: \"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.158307 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.158287 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q266w\" (UniqueName: \"kubernetes.io/projected/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-kube-api-access-q266w\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.158514 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.158498 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9fp54\" (UniqueName: \"kubernetes.io/projected/be0b314d-538c-4868-a86d-72964ae80f4a-kube-api-access-9fp54\") pod \"iptables-alerter-dmjpb\" (UID: \"be0b314d-538c-4868-a86d-72964ae80f4a\") " pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.159371 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.159356 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bglvj\" (UniqueName: \"kubernetes.io/projected/9afa5d28-b5ca-4780-8f04-89166c9b1054-kube-api-access-bglvj\") pod \"node-ca-7t9s9\" (UID: \"9afa5d28-b5ca-4780-8f04-89166c9b1054\") " pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.174522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.174503 2570 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:32.249121 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249092 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-tmp\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249225 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249141 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-host\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249225 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249159 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysconfig\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249225 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249175 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-lib-modules\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249369 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249229 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-host\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249369 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249277 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-lib-modules\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249369 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249288 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysconfig\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249369 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249314 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-conf\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249369 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249355 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8cfqr\" (UniqueName: \"kubernetes.io/projected/472bb351-9e91-46f6-bcd5-fb53b06a2aad-kube-api-access-8cfqr\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249383 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-run\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249429 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-modprobe-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249445 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-conf\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249455 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249487 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-systemd\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249492 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-run\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249510 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-tuned\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249533 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-systemd\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249538 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-kubernetes\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249551 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-modprobe-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249560 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-sys\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249585 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-var-lib-kubelet\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.249601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249602 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-kubernetes\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.250023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249607 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-sysctl-d\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.250023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249619 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-sys\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.250023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.249666 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/472bb351-9e91-46f6-bcd5-fb53b06a2aad-var-lib-kubelet\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.251632 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.251614 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-tmp\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.251683 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.251641 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/472bb351-9e91-46f6-bcd5-fb53b06a2aad-etc-tuned\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.257273 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.257258 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8cfqr\" (UniqueName: \"kubernetes.io/projected/472bb351-9e91-46f6-bcd5-fb53b06a2aad-kube-api-access-8cfqr\") pod \"tuned-wf69v\" (UID: \"472bb351-9e91-46f6-bcd5-fb53b06a2aad\") " pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.368467 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.368359 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" Jun 02 21:33:32.374540 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.374513 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:32.374678 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.374648 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod291f68dc_768f_463f_8177_f9b25147390e.slice/crio-4aa0b1ef24348f30e3d7fc5ce196b236b5b7a44b500c9f38d311932df19fc7a3 WatchSource:0}: Error finding container 4aa0b1ef24348f30e3d7fc5ce196b236b5b7a44b500c9f38d311932df19fc7a3: Status 404 returned error can't find the container with id 4aa0b1ef24348f30e3d7fc5ce196b236b5b7a44b500c9f38d311932df19fc7a3 Jun 02 21:33:32.380269 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.380243 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc682cb62_1356_4ee4_ad9a_f7a368bdba49.slice/crio-f5dcb4d18a49c28803eec5d039b004bc05c020df8b83db1797426329b224603d WatchSource:0}: Error finding container f5dcb4d18a49c28803eec5d039b004bc05c020df8b83db1797426329b224603d: Status 404 returned error can't find the container with id f5dcb4d18a49c28803eec5d039b004bc05c020df8b83db1797426329b224603d Jun 02 21:33:32.382081 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.381979 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:32.386982 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.386960 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" Jun 02 21:33:32.387220 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.387193 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod10581f86_1587_4d24_97fc_7630c1309315.slice/crio-e412f1374f43043f03d813a631aa6e84f152386545d140a2afa7e0b2d32062d2 WatchSource:0}: Error finding container e412f1374f43043f03d813a631aa6e84f152386545d140a2afa7e0b2d32062d2: Status 404 returned error can't find the container with id e412f1374f43043f03d813a631aa6e84f152386545d140a2afa7e0b2d32062d2 Jun 02 21:33:32.392255 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.392238 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-7t9s9" Jun 02 21:33:32.393110 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.393091 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf8922dba_eb5d_4c0d_b629_7f4ac7a94bf8.slice/crio-b0bffc18fd758977b0473174cc6060630c071c7448075a856f3b5a7d0943d641 WatchSource:0}: Error finding container b0bffc18fd758977b0473174cc6060630c071c7448075a856f3b5a7d0943d641: Status 404 returned error can't find the container with id b0bffc18fd758977b0473174cc6060630c071c7448075a856f3b5a7d0943d641 Jun 02 21:33:32.397225 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.397208 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-wbmzn" Jun 02 21:33:32.398418 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.398398 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9afa5d28_b5ca_4780_8f04_89166c9b1054.slice/crio-68e6d14cd47594011f77b10e5067b63df37f9a1e033e6cca14733c9b83d9a454 WatchSource:0}: Error finding container 68e6d14cd47594011f77b10e5067b63df37f9a1e033e6cca14733c9b83d9a454: Status 404 returned error can't find the container with id 68e6d14cd47594011f77b10e5067b63df37f9a1e033e6cca14733c9b83d9a454 Jun 02 21:33:32.403020 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.403000 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-dmjpb" Jun 02 21:33:32.403520 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.403386 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod76b9dfcd_868b_46a7_8e10_4b624786b63e.slice/crio-27ed7bb3b9ab107c2ffa370b2bbe844f89b0e29bfe6d2e8c2a0093bb1a07a35e WatchSource:0}: Error finding container 27ed7bb3b9ab107c2ffa370b2bbe844f89b0e29bfe6d2e8c2a0093bb1a07a35e: Status 404 returned error can't find the container with id 27ed7bb3b9ab107c2ffa370b2bbe844f89b0e29bfe6d2e8c2a0093bb1a07a35e Jun 02 21:33:32.407277 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.407256 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-wf69v" Jun 02 21:33:32.408617 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.408601 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbe0b314d_538c_4868_a86d_72964ae80f4a.slice/crio-6550f19b25f3021082603d022bdb33205d75f56fce466a35f6d25d0ae0d3f0fe WatchSource:0}: Error finding container 6550f19b25f3021082603d022bdb33205d75f56fce466a35f6d25d0ae0d3f0fe: Status 404 returned error can't find the container with id 6550f19b25f3021082603d022bdb33205d75f56fce466a35f6d25d0ae0d3f0fe Jun 02 21:33:32.413953 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:32.413921 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod472bb351_9e91_46f6_bcd5_fb53b06a2aad.slice/crio-019e9b79c9ecd0575fe10a5f2d8d039e36737c8b5eab0c7d12d8371a66b020de WatchSource:0}: Error finding container 019e9b79c9ecd0575fe10a5f2d8d039e36737c8b5eab0c7d12d8371a66b020de: Status 404 returned error can't find the container with id 019e9b79c9ecd0575fe10a5f2d8d039e36737c8b5eab0c7d12d8371a66b020de Jun 02 21:33:32.633477 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.633392 2570 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:32.651962 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.651930 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:32.652139 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.652119 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:32.652213 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.652189 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:33:33.652170282 +0000 UTC m=+3.078730751 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:32.753220 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.753137 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:32.753369 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.753266 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:32.753369 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.753288 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:32.753369 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.753301 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:32.753369 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:32.753358 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:33:33.753337461 +0000 UTC m=+3.179897936 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:32.877250 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:32.877029 2570 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:33.088871 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.088775 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:28:32 +0000 UTC" deadline="2028-02-06 08:24:48.612805663 +0000 UTC" Jun 02 21:33:33.088871 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.088822 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="14722h51m15.523987978s" Jun 02 21:33:33.140180 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.140151 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:33.140335 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.140281 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:33.150011 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.149984 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7w995" event={"ID":"10581f86-1587-4d24-97fc-7630c1309315","Type":"ContainerStarted","Data":"e412f1374f43043f03d813a631aa6e84f152386545d140a2afa7e0b2d32062d2"} Jun 02 21:33:33.154434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.154395 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerStarted","Data":"4aa0b1ef24348f30e3d7fc5ce196b236b5b7a44b500c9f38d311932df19fc7a3"} Jun 02 21:33:33.159667 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.159645 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wf69v" event={"ID":"472bb351-9e91-46f6-bcd5-fb53b06a2aad","Type":"ContainerStarted","Data":"019e9b79c9ecd0575fe10a5f2d8d039e36737c8b5eab0c7d12d8371a66b020de"} Jun 02 21:33:33.162116 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.162087 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dmjpb" event={"ID":"be0b314d-538c-4868-a86d-72964ae80f4a","Type":"ContainerStarted","Data":"6550f19b25f3021082603d022bdb33205d75f56fce466a35f6d25d0ae0d3f0fe"} Jun 02 21:33:33.167522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.167498 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbmzn" event={"ID":"76b9dfcd-868b-46a7-8e10-4b624786b63e","Type":"ContainerStarted","Data":"27ed7bb3b9ab107c2ffa370b2bbe844f89b0e29bfe6d2e8c2a0093bb1a07a35e"} Jun 02 21:33:33.181986 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.181959 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7t9s9" event={"ID":"9afa5d28-b5ca-4780-8f04-89166c9b1054","Type":"ContainerStarted","Data":"68e6d14cd47594011f77b10e5067b63df37f9a1e033e6cca14733c9b83d9a454"} Jun 02 21:33:33.191528 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.191502 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" event={"ID":"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8","Type":"ContainerStarted","Data":"b0bffc18fd758977b0473174cc6060630c071c7448075a856f3b5a7d0943d641"} Jun 02 21:33:33.201244 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.201195 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"f5dcb4d18a49c28803eec5d039b004bc05c020df8b83db1797426329b224603d"} Jun 02 21:33:33.662780 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.662137 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:33.662780 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.662293 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:33.662780 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.662354 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:33:35.662333605 +0000 UTC m=+5.088894095 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:33.763378 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.762727 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:33.763378 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.762918 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:33.763378 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.762935 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:33.763378 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.762948 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:33.763378 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:33.763003 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:33:35.762982967 +0000 UTC m=+5.189543458 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:33.821882 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:33.821688 2570 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160" Jun 02 21:33:34.089392 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:34.089331 2570 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-06-01 21:28:32 +0000 UTC" deadline="2027-11-11 11:05:07.253214487 +0000 UTC" Jun 02 21:33:34.089392 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:34.089365 2570 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12637h31m33.163853391s" Jun 02 21:33:34.137009 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:34.136961 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:34.137198 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:34.137171 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:35.138062 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:35.137579 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:35.138062 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.137708 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:35.677616 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:35.677065 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:35.677616 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.677235 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:35.677616 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.677293 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:33:39.677275038 +0000 UTC m=+9.103835521 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:35.778282 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:35.778249 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:35.778449 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.778411 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:35.778449 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.778430 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:35.778449 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.778442 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:35.778613 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:35.778497 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:33:39.778479594 +0000 UTC m=+9.205040069 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:36.136903 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:36.136870 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:36.137104 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:36.136992 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:37.137660 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.137627 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:37.138149 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:37.137768 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:37.600535 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.598076 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-bsngs"] Jun 02 21:33:37.601936 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.601913 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.604809 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.604790 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\"" Jun 02 21:33:37.604809 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.604796 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\"" Jun 02 21:33:37.604984 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.604806 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rvkzg\"" Jun 02 21:33:37.696382 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.696354 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-hosts-file\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.696513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.696423 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hdjbx\" (UniqueName: \"kubernetes.io/projected/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-kube-api-access-hdjbx\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.696513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.696450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-tmp-dir\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.797591 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.797557 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-hosts-file\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.797733 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.797637 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hdjbx\" (UniqueName: \"kubernetes.io/projected/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-kube-api-access-hdjbx\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.797733 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.797665 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-tmp-dir\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.797733 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.797675 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-hosts-file\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.798028 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.798001 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-tmp-dir\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.806424 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.806402 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hdjbx\" (UniqueName: \"kubernetes.io/projected/cdf0cb33-6a7d-4973-8c72-6292c5710ef9-kube-api-access-hdjbx\") pod \"node-resolver-bsngs\" (UID: \"cdf0cb33-6a7d-4973-8c72-6292c5710ef9\") " pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:37.911206 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:37.911135 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-bsngs" Jun 02 21:33:38.137574 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:38.137545 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:38.137742 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:38.137667 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:39.137325 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:39.137281 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:39.137489 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.137420 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:39.711958 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:39.711924 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:39.712345 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.712041 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:39.712345 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.712114 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:33:47.712094756 +0000 UTC m=+17.138655221 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:39.812933 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:39.812897 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:39.813125 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.813038 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:39.813125 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.813069 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:39.813125 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.813081 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:39.813298 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:39.813143 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:33:47.813122162 +0000 UTC m=+17.239682627 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:40.137596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:40.137521 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:40.137759 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:40.137663 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:41.137183 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:41.137156 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:41.137623 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:41.137254 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:42.137215 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:42.137178 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:42.137647 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:42.137298 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:43.137474 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.137438 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:43.137927 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:43.137581 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:43.486791 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.486754 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-mz9vm"] Jun 02 21:33:43.517445 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.517423 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.517556 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:43.517507 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:43.643132 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.643098 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-kubelet-config\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.643307 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.643149 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.643307 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.643225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-dbus\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.743839 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.743769 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-kubelet-config\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.743839 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.743807 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.743839 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.743836 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-dbus\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.744065 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.743916 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-dbus\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.744065 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:43.743933 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:43.744065 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:43.743955 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-kubelet-config\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:43.744065 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:43.743996 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:44.243981967 +0000 UTC m=+13.670542433 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:44.137247 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:44.137175 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:44.137395 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:44.137294 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:44.248289 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:44.248259 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:44.248697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:44.248394 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:44.248697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:44.248454 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:45.248436515 +0000 UTC m=+14.674996990 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:45.136832 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:45.136792 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:45.136997 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:45.136849 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:45.136997 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:45.136933 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:45.137159 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:45.137086 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:45.257354 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:45.257314 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:45.257739 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:45.257427 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:45.257739 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:45.257506 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:47.257485758 +0000 UTC m=+16.684046226 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:46.137243 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:46.137211 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:46.137378 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:46.137306 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:47.137488 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:47.137454 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:47.137957 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:47.137454 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:47.137957 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.137589 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:47.137957 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.137747 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:47.273188 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:47.273151 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:47.273336 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.273318 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:47.273405 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.273387 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:51.273371345 +0000 UTC m=+20.699931811 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:47.777320 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:47.777270 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:47.777539 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.777445 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:47.777539 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.777526 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:34:03.777503914 +0000 UTC m=+33.204064381 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:33:47.878111 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:47.878067 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:47.878279 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.878245 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:33:47.878332 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.878273 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:33:47.878332 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.878302 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:47.878405 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:47.878367 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:34:03.87834675 +0000 UTC m=+33.304907219 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:33:48.137329 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:48.137252 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:48.137489 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:48.137367 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:49.137706 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:49.137221 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:49.137706 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:49.137221 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:49.137706 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:49.137348 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:49.137706 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:49.137406 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:50.137100 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:50.137069 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:50.137267 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:50.137185 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:50.657389 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:33:50.657325 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcdf0cb33_6a7d_4973_8c72_6292c5710ef9.slice/crio-55d30a929fda476d9c08f0c04ac648152d6362e0307c96e1811ee7141a7b38b5 WatchSource:0}: Error finding container 55d30a929fda476d9c08f0c04ac648152d6362e0307c96e1811ee7141a7b38b5: Status 404 returned error can't find the container with id 55d30a929fda476d9c08f0c04ac648152d6362e0307c96e1811ee7141a7b38b5 Jun 02 21:33:51.138414 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.138087 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:51.138564 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.138099 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:51.138671 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:51.138647 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:51.138704 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:51.138688 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:51.236836 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.236787 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" event={"ID":"a0fa59b950aacfeb2c5878ddb9277dec","Type":"ContainerStarted","Data":"b6a76eefef38023ee51026b6efb98c71ba0dc523c1957451ba1227a2d08b52c9"} Jun 02 21:33:51.238519 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.238491 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-7w995" event={"ID":"10581f86-1587-4d24-97fc-7630c1309315","Type":"ContainerStarted","Data":"4b73777564a88dba6067e016bd7a99fe7092992f3b50a40dcbf2092a333e554f"} Jun 02 21:33:51.240669 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.240639 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerStarted","Data":"365b0d831c45c1305cd5f4bc39702f983fe38c202f5135ba55b959c064c9f58c"} Jun 02 21:33:51.242658 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.242631 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bsngs" event={"ID":"cdf0cb33-6a7d-4973-8c72-6292c5710ef9","Type":"ContainerStarted","Data":"3ab5547f1fe723e1e0ac24ad55d42440bb8adb7e2c7da68f6f50567d718700de"} Jun 02 21:33:51.242757 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.242675 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-bsngs" event={"ID":"cdf0cb33-6a7d-4973-8c72-6292c5710ef9","Type":"ContainerStarted","Data":"55d30a929fda476d9c08f0c04ac648152d6362e0307c96e1811ee7141a7b38b5"} Jun 02 21:33:51.243931 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.243912 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" event={"ID":"51bf9710c49ae2c5fd873f7bcc330dcd","Type":"ContainerStarted","Data":"dedaa3fa65a53ffd3b593ec88365b705c652bcd807735886167a723430c4065b"} Jun 02 21:33:51.245345 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.245324 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-wf69v" event={"ID":"472bb351-9e91-46f6-bcd5-fb53b06a2aad","Type":"ContainerStarted","Data":"767cc74695d376c22f7e9ed22bfa8d3bce3a54ca221d512a9af496c4c46de1a8"} Jun 02 21:33:51.246756 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.246729 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-wbmzn" event={"ID":"76b9dfcd-868b-46a7-8e10-4b624786b63e","Type":"ContainerStarted","Data":"242ecdabe86a8402f1b85d840be59827e0a8aaa063084265e5e283baf447d843"} Jun 02 21:33:51.248555 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.248526 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-7t9s9" event={"ID":"9afa5d28-b5ca-4780-8f04-89166c9b1054","Type":"ContainerStarted","Data":"6c308ed1fe81c1441a5c9200c8b67c3fa83b8af15de96bcad520420a62d8668f"} Jun 02 21:33:51.249972 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.249940 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" event={"ID":"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8","Type":"ContainerStarted","Data":"5114f241ae73bb6061e755fe577641d5d835a1a1a676c2174206a2be908eda91"} Jun 02 21:33:51.251830 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.251810 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:33:51.252287 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.252249 2570 generic.go:358] "Generic (PLEG): container finished" podID="c682cb62-1356-4ee4-ad9a-f7a368bdba49" containerID="cd3b796bc2919046c6df004f4af31e6af298b50bdedfb770c4f23cfd8e601ee8" exitCode=1 Jun 02 21:33:51.252287 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.252285 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"51c9e5a451868ded3734800a80054c97d9706450a5a1bbc2d98023d2167587ee"} Jun 02 21:33:51.252472 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.252305 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"038ba3946cff18f606a2b870effdfb4739c931c896cb63003abd1e579af6def2"} Jun 02 21:33:51.252472 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.252318 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerDied","Data":"cd3b796bc2919046c6df004f4af31e6af298b50bdedfb770c4f23cfd8e601ee8"} Jun 02 21:33:51.252472 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.252328 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"e3de66f10e0023d76ff85b09e8fde4a92e36ad6a0f8421a03b68007022432ce8"} Jun 02 21:33:51.278842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.278796 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-7w995" podStartSLOduration=2.275899812 podStartE2EDuration="20.278783676s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.388841939 +0000 UTC m=+1.815402419" lastFinishedPulling="2026-06-02 21:33:50.391725814 +0000 UTC m=+19.818286283" observedRunningTime="2026-06-02 21:33:51.278528573 +0000 UTC m=+20.705089061" watchObservedRunningTime="2026-06-02 21:33:51.278783676 +0000 UTC m=+20.705344163" Jun 02 21:33:51.296023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.295988 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-wbmzn" podStartSLOduration=1.891008374 podStartE2EDuration="20.29597648s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.405921543 +0000 UTC m=+1.832482009" lastFinishedPulling="2026-06-02 21:33:50.810889638 +0000 UTC m=+20.237450115" observedRunningTime="2026-06-02 21:33:51.295791818 +0000 UTC m=+20.722352307" watchObservedRunningTime="2026-06-02 21:33:51.29597648 +0000 UTC m=+20.722536968" Jun 02 21:33:51.306150 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.306125 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:51.306284 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:51.306265 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:51.306365 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:51.306326 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:33:59.30631063 +0000 UTC m=+28.732871115 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:51.308943 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.308913 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-wf69v" podStartSLOduration=2.002930806 podStartE2EDuration="20.308903255s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.416394681 +0000 UTC m=+1.842955146" lastFinishedPulling="2026-06-02 21:33:50.722367114 +0000 UTC m=+20.148927595" observedRunningTime="2026-06-02 21:33:51.30860471 +0000 UTC m=+20.735165198" watchObservedRunningTime="2026-06-02 21:33:51.308903255 +0000 UTC m=+20.735463744" Jun 02 21:33:51.322400 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.320687 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-bsngs" podStartSLOduration=14.32067334 podStartE2EDuration="14.32067334s" podCreationTimestamp="2026-06-02 21:33:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:33:51.319780512 +0000 UTC m=+20.746341000" watchObservedRunningTime="2026-06-02 21:33:51.32067334 +0000 UTC m=+20.747233828" Jun 02 21:33:51.356823 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.356790 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-134-230.ec2.internal" podStartSLOduration=20.356778998 podStartE2EDuration="20.356778998s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:33:51.33436297 +0000 UTC m=+20.760923459" watchObservedRunningTime="2026-06-02 21:33:51.356778998 +0000 UTC m=+20.783339485" Jun 02 21:33:51.356906 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:51.356887 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-7t9s9" podStartSLOduration=10.195448346 podStartE2EDuration="20.35688315s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.400365292 +0000 UTC m=+1.826925762" lastFinishedPulling="2026-06-02 21:33:42.561800087 +0000 UTC m=+11.988360566" observedRunningTime="2026-06-02 21:33:51.356525362 +0000 UTC m=+20.783085849" watchObservedRunningTime="2026-06-02 21:33:51.35688315 +0000 UTC m=+20.783443637" Jun 02 21:33:52.133354 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.133194 2570 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock" Jun 02 21:33:52.137256 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.137236 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:52.137372 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:52.137349 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:52.255054 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.255011 2570 generic.go:358] "Generic (PLEG): container finished" podID="a0fa59b950aacfeb2c5878ddb9277dec" containerID="b6a76eefef38023ee51026b6efb98c71ba0dc523c1957451ba1227a2d08b52c9" exitCode=0 Jun 02 21:33:52.255199 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.255099 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" event={"ID":"a0fa59b950aacfeb2c5878ddb9277dec","Type":"ContainerDied","Data":"b6a76eefef38023ee51026b6efb98c71ba0dc523c1957451ba1227a2d08b52c9"} Jun 02 21:33:52.256484 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.256461 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="365b0d831c45c1305cd5f4bc39702f983fe38c202f5135ba55b959c064c9f58c" exitCode=0 Jun 02 21:33:52.256588 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.256530 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"365b0d831c45c1305cd5f4bc39702f983fe38c202f5135ba55b959c064c9f58c"} Jun 02 21:33:52.257881 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.257807 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-dmjpb" event={"ID":"be0b314d-538c-4868-a86d-72964ae80f4a","Type":"ContainerStarted","Data":"8d9d5bf0ab75e12b0fe995f06d067f9d57e2749f3a20aabe5cf779db4bfd9c0c"} Jun 02 21:33:52.259588 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.259565 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" event={"ID":"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8","Type":"ContainerStarted","Data":"bb1e0cdfbb1f231027ad397057a7a8cf940df9ae1d223dc0554dec347e356c4e"} Jun 02 21:33:52.261875 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.261861 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:33:52.262233 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.262211 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"fa4ce5ce6d19912abc4c88fc0a63b7cd4351ae0ece1dd1aa0306eaa99615643a"} Jun 02 21:33:52.262309 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.262241 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"acbb25eb91a865423f2899069fc603ba81b4280dde3f0e7640ccef5424f28b3d"} Jun 02 21:33:52.279109 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:52.279074 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-dmjpb" podStartSLOduration=3.298418897 podStartE2EDuration="21.279059004s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.411075422 +0000 UTC m=+1.837635908" lastFinishedPulling="2026-06-02 21:33:50.39171553 +0000 UTC m=+19.818276015" observedRunningTime="2026-06-02 21:33:52.278378008 +0000 UTC m=+21.704938497" watchObservedRunningTime="2026-06-02 21:33:52.279059004 +0000 UTC m=+21.705619490" Jun 02 21:33:53.115379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.115207 2570 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-06-02T21:33:52.133350283Z","UUID":"89518fcb-4f92-4303-9511-a611122d95e1","Handler":null,"Name":"","Endpoint":""} Jun 02 21:33:53.118450 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.118418 2570 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0 Jun 02 21:33:53.118450 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.118446 2570 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock Jun 02 21:33:53.136896 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.136872 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:53.137442 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:53.136992 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:53.137442 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.137068 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:53.137442 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:53.137186 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:53.266616 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.266518 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" event={"ID":"f8922dba-eb5d-4c0d-b629-7f4ac7a94bf8","Type":"ContainerStarted","Data":"50a5da0414a7d2f076e053ef34d17d17efe180176c35035e422f7fcb4be081e6"} Jun 02 21:33:53.268367 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.268337 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" event={"ID":"a0fa59b950aacfeb2c5878ddb9277dec","Type":"ContainerStarted","Data":"2e69e9706864c402948b53ea078e33c67c4f4c1b2ce4d7a2415e4c9541e0e31e"} Jun 02 21:33:53.295406 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.295371 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-ksk6w" podStartSLOduration=1.722582378 podStartE2EDuration="22.295359329s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.396146955 +0000 UTC m=+1.822707421" lastFinishedPulling="2026-06-02 21:33:52.968923903 +0000 UTC m=+22.395484372" observedRunningTime="2026-06-02 21:33:53.29375465 +0000 UTC m=+22.720315138" watchObservedRunningTime="2026-06-02 21:33:53.295359329 +0000 UTC m=+22.721919796" Jun 02 21:33:53.308110 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:53.308082 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-134-230.ec2.internal" podStartSLOduration=22.308072312 podStartE2EDuration="22.308072312s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 21:33:53.307527432 +0000 UTC m=+22.734087919" watchObservedRunningTime="2026-06-02 21:33:53.308072312 +0000 UTC m=+22.734632797" Jun 02 21:33:54.137578 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:54.137546 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:54.138101 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:54.137655 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:54.273288 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:54.273261 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:33:54.273931 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:54.273901 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"554021d761acf0eecf8321272e7205aefa05e8fc06ade54aac993c2944c22769"} Jun 02 21:33:54.450402 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:54.450334 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:54.450985 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:54.450958 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:55.136581 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:55.136556 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:55.136761 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:55.136672 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:55.136894 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:55.136869 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:55.137016 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:55.136983 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:56.137115 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.136890 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:56.137742 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:56.137207 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:56.279635 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.279610 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:33:56.279973 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.279950 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"ceb09bde0caf3c35da7a122bc0dec8b239166e90d32a733e146afc58a8a6f5cc"} Jun 02 21:33:56.280375 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.280354 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:56.280456 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.280385 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:56.280506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.280483 2570 scope.go:117] "RemoveContainer" containerID="cd3b796bc2919046c6df004f4af31e6af298b50bdedfb770c4f23cfd8e601ee8" Jun 02 21:33:56.297652 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:56.297630 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:57.136689 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.136660 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:57.136689 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.136680 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:57.136921 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:57.136772 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:57.136972 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:57.136945 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:57.779561 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.778189 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-scmcl"] Jun 02 21:33:57.779561 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.778467 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:33:57.779561 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:57.778549 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:33:57.781544 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.781521 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vq7rr"] Jun 02 21:33:57.781657 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.781602 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:57.781743 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:57.781680 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:57.792501 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.792478 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mz9vm"] Jun 02 21:33:57.792598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:57.792542 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:57.792635 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:57.792604 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:58.286228 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.286206 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:33:58.286514 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.286492 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" event={"ID":"c682cb62-1356-4ee4-ad9a-f7a368bdba49","Type":"ContainerStarted","Data":"1d253c3f3d23cdf01062714a163b48c277250a8b42b15deb42baa0227055d2c3"} Jun 02 21:33:58.286775 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.286762 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:58.288120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.288091 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="0d46c10fb5117b3e9c9ed11f96d6ea26ad4ac078cbb155358736fa7537408951" exitCode=0 Jun 02 21:33:58.288203 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.288141 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"0d46c10fb5117b3e9c9ed11f96d6ea26ad4ac078cbb155358736fa7537408951"} Jun 02 21:33:58.301404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.301384 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:33:58.314683 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:58.314090 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" podStartSLOduration=8.935935891 podStartE2EDuration="27.314073655s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.381761108 +0000 UTC m=+1.808321577" lastFinishedPulling="2026-06-02 21:33:50.75989887 +0000 UTC m=+20.186459341" observedRunningTime="2026-06-02 21:33:58.313460551 +0000 UTC m=+27.740021039" watchObservedRunningTime="2026-06-02 21:33:58.314073655 +0000 UTC m=+27.740634147" Jun 02 21:33:59.082847 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.082817 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:59.083284 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.082936 2570 prober_manager.go:312] "Failed to trigger a manual run" probe="Readiness" Jun 02 21:33:59.083394 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.083378 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-7w995" Jun 02 21:33:59.137116 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.137082 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:33:59.137272 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.137082 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:59.137272 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:59.137201 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:33:59.137272 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:59.137252 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:33:59.368948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:33:59.368915 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:33:59.369181 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:59.369132 2570 secret.go:189] Couldn't get secret kube-system/original-pull-secret: object "kube-system"/"original-pull-secret" not registered Jun 02 21:33:59.369285 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:33:59.369205 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret podName:81fe2e5d-3e81-4d24-b43e-52ab19e21e86 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:15.369171485 +0000 UTC m=+44.795731971 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "original-pull-secret" (UniqueName: "kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret") pod "global-pull-secret-syncer-mz9vm" (UID: "81fe2e5d-3e81-4d24-b43e-52ab19e21e86") : object "kube-system"/"original-pull-secret" not registered Jun 02 21:34:00.137485 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:00.137457 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:00.138000 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:00.137558 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:34:00.293687 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:00.293657 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="2d76cafd6be3b15e008ac8a8b4d3948eb75572af14d46a42eca338ffb603dfce" exitCode=0 Jun 02 21:34:00.293851 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:00.293743 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"2d76cafd6be3b15e008ac8a8b4d3948eb75572af14d46a42eca338ffb603dfce"} Jun 02 21:34:00.309533 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:00.309484 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" podUID="c682cb62-1356-4ee4-ad9a-f7a368bdba49" containerName="ovnkube-controller" probeResult="failure" output="" Jun 02 21:34:01.138755 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:01.138733 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:34:01.139380 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:01.138852 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:34:01.139380 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:01.138888 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:01.139380 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:01.138964 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:34:01.297691 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:01.297652 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="8e968d16629958de7e5c37a002e185a868ed55c646836814dfd3817cd9765818" exitCode=0 Jun 02 21:34:01.297836 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:01.297714 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"8e968d16629958de7e5c37a002e185a868ed55c646836814dfd3817cd9765818"} Jun 02 21:34:02.137197 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:02.137165 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:02.137334 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:02.137285 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-scmcl" podUID="2f994d2c-9570-4f84-88b2-53c4d15e5cda" Jun 02 21:34:03.137161 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.137131 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:03.137161 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.137156 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:34:03.137656 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.137248 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="kube-system/global-pull-secret-syncer-mz9vm" podUID="81fe2e5d-3e81-4d24-b43e-52ab19e21e86" Jun 02 21:34:03.137656 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.137380 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:34:03.806304 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.806272 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:34:03.806453 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.806384 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:03.806453 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.806445 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.806426653 +0000 UTC m=+65.232987121 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : object "openshift-multus"/"metrics-daemon-secret" not registered Jun 02 21:34:03.907366 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.907331 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:03.907503 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.907473 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered Jun 02 21:34:03.907503 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.907490 2570 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered Jun 02 21:34:03.907503 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.907498 2570 projected.go:194] Error preparing data for projected volume kube-api-access-clgrn for pod openshift-network-diagnostics/network-check-target-scmcl: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:03.907596 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:03.907543 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn podName:2f994d2c-9570-4f84-88b2-53c4d15e5cda nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.907530191 +0000 UTC m=+65.334090657 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-clgrn" (UniqueName: "kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn") pod "network-check-target-scmcl" (UID: "2f994d2c-9570-4f84-88b2-53c4d15e5cda") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered] Jun 02 21:34:03.933495 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.933477 2570 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-134-230.ec2.internal" event="NodeReady" Jun 02 21:34:03.933622 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.933561 2570 kubelet_node_status.go:550] "Fast updating node status as it just became ready" Jun 02 21:34:03.972913 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.972884 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c"] Jun 02 21:34:03.976521 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.976502 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k"] Jun 02 21:34:03.976645 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.976626 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:03.978697 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.978681 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\"" Jun 02 21:34:03.978773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.978711 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-bjxvc\"" Jun 02 21:34:03.979256 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.979238 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\"" Jun 02 21:34:03.979348 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.979238 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\"" Jun 02 21:34:03.979348 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.979274 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\"" Jun 02 21:34:03.979451 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.979434 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-9c855867d-6fdx9"] Jun 02 21:34:03.979566 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.979553 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:03.981928 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.981912 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2xlld\"" Jun 02 21:34:03.982119 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.982098 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf"] Jun 02 21:34:03.982227 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.982165 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\"" Jun 02 21:34:03.982284 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.982261 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:03.985844 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.984984 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7"] Jun 02 21:34:03.985844 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.985157 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:03.985844 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.985225 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\"" Jun 02 21:34:03.985844 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.985780 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\"" Jun 02 21:34:03.986150 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.985941 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\"" Jun 02 21:34:03.986150 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.985963 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-x4k7h\"" Jun 02 21:34:03.986386 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.986367 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\"" Jun 02 21:34:03.987898 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.987883 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-pnh9b"] Jun 02 21:34:03.988023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.988011 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:03.993083 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.993033 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c"] Jun 02 21:34:03.993205 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.993185 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:03.994000 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.993980 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-ca\"" Jun 02 21:34:03.994143 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.994123 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-open-cluster-management.io-proxy-agent-signer-client-cert\"" Jun 02 21:34:03.994262 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.994245 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-service-proxy-server-certificates\"" Jun 02 21:34:03.994430 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.994414 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"cluster-proxy-hub-kubeconfig\"" Jun 02 21:34:03.994740 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.994719 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"work-manager-hub-kubeconfig\"" Jun 02 21:34:03.995359 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.995333 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k"] Jun 02 21:34:03.997328 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.997287 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gs96v\"" Jun 02 21:34:03.997566 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.997526 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\"" Jun 02 21:34:03.998770 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.998552 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf"] Jun 02 21:34:03.999320 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.999296 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7"] Jun 02 21:34:03.999716 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:03.999693 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\"" Jun 02 21:34:04.000522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.000505 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\"" Jun 02 21:34:04.033401 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.033380 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-gzfbq"] Jun 02 21:34:04.036507 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.036487 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pnh9b"] Jun 02 21:34:04.036653 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.036514 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-9c855867d-6fdx9"] Jun 02 21:34:04.036653 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.036615 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.039713 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.039693 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\"" Jun 02 21:34:04.039824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.039806 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2lz6l\"" Jun 02 21:34:04.040288 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.040252 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\"" Jun 02 21:34:04.042911 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.042888 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\"" Jun 02 21:34:04.058003 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.057959 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gzfbq"] Jun 02 21:34:04.108884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108850 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.108995 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108892 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.108995 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108919 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s9t55\" (UniqueName: \"kubernetes.io/projected/994dfd6d-3332-447f-869d-f4353d792e2c-kube-api-access-s9t55\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.108995 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108949 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc1183-d347-459e-8443-709b14b46269-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.108995 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108973 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.108996 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109023 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109067 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/994dfd6d-3332-447f-869d-f4353d792e2c-config-volume\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109114 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.109259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109223 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109271 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994dfd6d-3332-447f-869d-f4353d792e2c-tmp-dir\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109351 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a3a47-73a4-443c-b727-2cdfd747d117-tmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109379 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mfj4d\" (UniqueName: \"kubernetes.io/projected/c3dc1183-d347-459e-8443-709b14b46269-kube-api-access-mfj4d\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109401 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6d2b8be-500a-44f6-9ce7-9a031012335b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109425 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q2kw5\" (UniqueName: \"kubernetes.io/projected/e6d2b8be-500a-44f6-9ce7-9a031012335b-kube-api-access-q2kw5\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109465 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109492 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvjmp\" (UniqueName: \"kubernetes.io/projected/ac7a3a47-73a4-443c-b727-2cdfd747d117-kube-api-access-vvjmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109520 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9aa85708-a571-4b8c-862b-f77475fb5a2a-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.109549 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109538 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.109948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109580 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.109948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109621 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac7a3a47-73a4-443c-b727-2cdfd747d117-klusterlet-config\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.109948 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.109664 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jshc2\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.136935 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.136912 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:04.139588 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.139571 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-txpc4\"" Jun 02 21:34:04.139882 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.139587 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:34:04.139882 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.139611 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:34:04.210709 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210676 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6d2b8be-500a-44f6-9ce7-9a031012335b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.210709 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210706 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.210979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210726 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.210979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210746 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.210979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210763 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9aa85708-a571-4b8c-862b-f77475fb5a2a-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.210979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210903 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jshc2\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.210979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.210941 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211010 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211037 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211080 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/994dfd6d-3332-447f-869d-f4353d792e2c-config-volume\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211102 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a3a47-73a4-443c-b727-2cdfd747d117-tmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211125 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mfj4d\" (UniqueName: \"kubernetes.io/projected/c3dc1183-d347-459e-8443-709b14b46269-kube-api-access-mfj4d\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.211231 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.211309 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:04.711283258 +0000 UTC m=+34.137843726 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:04.211404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211351 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211391 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q2kw5\" (UniqueName: \"kubernetes.io/projected/e6d2b8be-500a-44f6-9ce7-9a031012335b-kube-api-access-q2kw5\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211646 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/994dfd6d-3332-447f-869d-f4353d792e2c-config-volume\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211652 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211506 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/9aa85708-a571-4b8c-862b-f77475fb5a2a-nginx-conf\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211688 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4bd6h\" (UniqueName: \"kubernetes.io/projected/68ef55e0-fb04-4198-b200-179452490e9f-kube-api-access-4bd6h\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211723 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211521 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/ac7a3a47-73a4-443c-b727-2cdfd747d117-tmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211401 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.211740 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211790 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac7a3a47-73a4-443c-b727-2cdfd747d117-klusterlet-config\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.211802 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.211804 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:04.711786897 +0000 UTC m=+34.138347406 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211837 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211878 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s9t55\" (UniqueName: \"kubernetes.io/projected/994dfd6d-3332-447f-869d-f4353d792e2c-kube-api-access-s9t55\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211903 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211924 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc1183-d347-459e-8443-709b14b46269-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211949 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.211970 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212000 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vvjmp\" (UniqueName: \"kubernetes.io/projected/ac7a3a47-73a4-443c-b727-2cdfd747d117-kube-api-access-vvjmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212026 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212068 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994dfd6d-3332-447f-869d-f4353d792e2c-tmp-dir\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.212190 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.212203 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.212293 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:04.712276456 +0000 UTC m=+34.138836935 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:04.212349 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212337 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/994dfd6d-3332-447f-869d-f4353d792e2c-tmp-dir\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.213003 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.212861 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.215208 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.215158 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ocpservice-ca\" (UniqueName: \"kubernetes.io/configmap/c3dc1183-d347-459e-8443-709b14b46269-ocpservice-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.216609 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.216940 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub-kubeconfig\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.217090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-hub\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.217099 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.217353 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-proxy-server-cert\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-service-proxy-server-cert\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.217372 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"klusterlet-config\" (UniqueName: \"kubernetes.io/secret/ac7a3a47-73a4-443c-b727-2cdfd747d117-klusterlet-config\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.216533 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca\" (UniqueName: \"kubernetes.io/secret/c3dc1183-d347-459e-8443-709b14b46269-ca\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.217434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.217372 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/e6d2b8be-500a-44f6-9ce7-9a031012335b-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.221659 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.221625 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.221809 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.221793 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jshc2\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.223372 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.223351 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s9t55\" (UniqueName: \"kubernetes.io/projected/994dfd6d-3332-447f-869d-f4353d792e2c-kube-api-access-s9t55\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.223766 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.223746 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q2kw5\" (UniqueName: \"kubernetes.io/projected/e6d2b8be-500a-44f6-9ce7-9a031012335b-kube-api-access-q2kw5\") pod \"managed-serviceaccount-addon-agent-74bc9b695b-5kl4c\" (UID: \"e6d2b8be-500a-44f6-9ce7-9a031012335b\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.225307 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.225289 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvjmp\" (UniqueName: \"kubernetes.io/projected/ac7a3a47-73a4-443c-b727-2cdfd747d117-kube-api-access-vvjmp\") pod \"klusterlet-addon-workmgr-597d74ff77-x2qmf\" (UID: \"ac7a3a47-73a4-443c-b727-2cdfd747d117\") " pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.225575 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.225555 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mfj4d\" (UniqueName: \"kubernetes.io/projected/c3dc1183-d347-459e-8443-709b14b46269-kube-api-access-mfj4d\") pod \"cluster-proxy-proxy-agent-78c7bd458-vkdx7\" (UID: \"c3dc1183-d347-459e-8443-709b14b46269\") " pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.298196 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.298168 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" Jun 02 21:34:04.313487 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.313432 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4bd6h\" (UniqueName: \"kubernetes.io/projected/68ef55e0-fb04-4198-b200-179452490e9f-kube-api-access-4bd6h\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.313649 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.313628 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.313844 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.313750 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:04.313844 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.313811 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:04.813793715 +0000 UTC m=+34.240354223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:04.320413 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.320393 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:04.325957 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.325928 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4bd6h\" (UniqueName: \"kubernetes.io/projected/68ef55e0-fb04-4198-b200-179452490e9f-kube-api-access-4bd6h\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.339792 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.339770 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:34:04.469660 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.469627 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c"] Jun 02 21:34:04.473929 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:34:04.473887 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pode6d2b8be_500a_44f6_9ce7_9a031012335b.slice/crio-c6f44921d92a1a8f14cc362f71fb7e20e17ec32e5aa4b15721719c030aced89c WatchSource:0}: Error finding container c6f44921d92a1a8f14cc362f71fb7e20e17ec32e5aa4b15721719c030aced89c: Status 404 returned error can't find the container with id c6f44921d92a1a8f14cc362f71fb7e20e17ec32e5aa4b15721719c030aced89c Jun 02 21:34:04.509707 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.509683 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf"] Jun 02 21:34:04.512386 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:34:04.512361 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac7a3a47_73a4_443c_b727_2cdfd747d117.slice/crio-4c17bc8b4b788fbb58f61ce7128a167864085c592e8de2e849e5367b34027acb WatchSource:0}: Error finding container 4c17bc8b4b788fbb58f61ce7128a167864085c592e8de2e849e5367b34027acb: Status 404 returned error can't find the container with id 4c17bc8b4b788fbb58f61ce7128a167864085c592e8de2e849e5367b34027acb Jun 02 21:34:04.535083 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.535039 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7"] Jun 02 21:34:04.537774 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:34:04.537748 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc3dc1183_d347_459e_8443_709b14b46269.slice/crio-2c64d6429dc3528831ccfd45288f917638808c6706eb521d027e7501557e8ebe WatchSource:0}: Error finding container 2c64d6429dc3528831ccfd45288f917638808c6706eb521d027e7501557e8ebe: Status 404 returned error can't find the container with id 2c64d6429dc3528831ccfd45288f917638808c6706eb521d027e7501557e8ebe Jun 02 21:34:04.716587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.716554 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:04.716710 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.716644 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:04.716757 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716702 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:04.716757 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.716712 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:04.716757 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716722 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:04.716882 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716775 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:04.716882 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716779 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:05.716757461 +0000 UTC m=+35.143317931 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:04.716882 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716818 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:04.716882 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716837 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:05.716816088 +0000 UTC m=+35.143376568 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:04.716882 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.716871 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:05.716857427 +0000 UTC m=+35.143417900 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:04.818040 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:04.818015 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:04.818165 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.818150 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:04.818212 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:04.818199 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:05.818187803 +0000 UTC m=+35.244748272 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:05.138112 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.137454 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:05.138112 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.137884 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:34:05.143324 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.143297 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4s2f5\"" Jun 02 21:34:05.143756 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.143591 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\"" Jun 02 21:34:05.146356 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.143854 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:34:05.310626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.310593 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" event={"ID":"ac7a3a47-73a4-443c-b727-2cdfd747d117","Type":"ContainerStarted","Data":"4c17bc8b4b788fbb58f61ce7128a167864085c592e8de2e849e5367b34027acb"} Jun 02 21:34:05.312981 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.312923 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" event={"ID":"e6d2b8be-500a-44f6-9ce7-9a031012335b","Type":"ContainerStarted","Data":"c6f44921d92a1a8f14cc362f71fb7e20e17ec32e5aa4b15721719c030aced89c"} Jun 02 21:34:05.319632 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.319606 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerStarted","Data":"2c64d6429dc3528831ccfd45288f917638808c6706eb521d027e7501557e8ebe"} Jun 02 21:34:05.725793 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.725756 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:05.725954 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.725849 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:05.725954 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.725901 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:05.726100 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726018 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:05.726100 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726032 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:05.726210 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726104 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:07.726085509 +0000 UTC m=+37.152645991 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:05.726515 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726496 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:05.726595 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726552 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:07.726537467 +0000 UTC m=+37.153097954 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:05.726653 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726608 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:05.726653 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.726640 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:07.726630675 +0000 UTC m=+37.153191145 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:05.827407 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:05.827312 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:05.827570 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.827470 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:05.827570 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:05.827526 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:07.827509982 +0000 UTC m=+37.254070501 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:07.743914 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:07.743876 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:07.743974 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:07.744039 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744090 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744165 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:11.744143984 +0000 UTC m=+41.170704458 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744175 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744100 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744190 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744224 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:11.744210765 +0000 UTC m=+41.170771241 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:07.744347 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.744239 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:11.744231284 +0000 UTC m=+41.170791755 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:07.845320 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:07.845282 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:07.845478 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.845419 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:07.845478 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:07.845478 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:11.84546067 +0000 UTC m=+41.272021135 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:11.775290 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:11.775085 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:11.775290 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775237 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:11.775290 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775257 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:11.775290 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:11.775285 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775305 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.775290279 +0000 UTC m=+49.201850745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775361 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:11.775370 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775396 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.775385856 +0000 UTC m=+49.201946321 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775433 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:11.775712 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.775462 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.775454828 +0000 UTC m=+49.202015294 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:11.876104 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:11.876075 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:11.876233 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.876193 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:11.876283 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:11.876235 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:19.876222808 +0000 UTC m=+49.302783274 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:15.339085 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.339028 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerStarted","Data":"235241ba00c0a2338c2f2ca065d95483794f6e009048205cc5c874bf8422d26c"} Jun 02 21:34:15.341166 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.341143 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="e365e85ae48781fe36fab2387cebc1016e4e244e9c75c41a8e714259d3b7902f" exitCode=0 Jun 02 21:34:15.341285 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.341198 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"e365e85ae48781fe36fab2387cebc1016e4e244e9c75c41a8e714259d3b7902f"} Jun 02 21:34:15.342450 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.342418 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" event={"ID":"ac7a3a47-73a4-443c-b727-2cdfd747d117","Type":"ContainerStarted","Data":"154445b44395bf41d38814df4cb62058f820a997fd5bbf7b757812111cbc7cf3"} Jun 02 21:34:15.342647 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.342623 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:15.343709 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.343684 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" event={"ID":"e6d2b8be-500a-44f6-9ce7-9a031012335b","Type":"ContainerStarted","Data":"59702d8136a46b0de84b00d943e65a09e95d2e74080cf9ffc8b2f84174f394bf"} Jun 02 21:34:15.344381 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.344363 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:34:15.376687 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.376629 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" podStartSLOduration=18.567145866 podStartE2EDuration="28.376609024s" podCreationTimestamp="2026-06-02 21:33:47 +0000 UTC" firstStartedPulling="2026-06-02 21:34:04.514003699 +0000 UTC m=+33.940564165" lastFinishedPulling="2026-06-02 21:34:14.323466844 +0000 UTC m=+43.750027323" observedRunningTime="2026-06-02 21:34:15.37562677 +0000 UTC m=+44.802187259" watchObservedRunningTime="2026-06-02 21:34:15.376609024 +0000 UTC m=+44.803169514" Jun 02 21:34:15.392912 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.392853 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" podStartSLOduration=18.545800064 podStartE2EDuration="28.39283231s" podCreationTimestamp="2026-06-02 21:33:47 +0000 UTC" firstStartedPulling="2026-06-02 21:34:04.476270522 +0000 UTC m=+33.902830988" lastFinishedPulling="2026-06-02 21:34:14.323302765 +0000 UTC m=+43.749863234" observedRunningTime="2026-06-02 21:34:15.391700282 +0000 UTC m=+44.818260770" watchObservedRunningTime="2026-06-02 21:34:15.39283231 +0000 UTC m=+44.819393001" Jun 02 21:34:15.404209 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.404166 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:15.407919 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.407891 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/81fe2e5d-3e81-4d24-b43e-52ab19e21e86-original-pull-secret\") pod \"global-pull-secret-syncer-mz9vm\" (UID: \"81fe2e5d-3e81-4d24-b43e-52ab19e21e86\") " pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:15.661107 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.661016 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-mz9vm" Jun 02 21:34:15.789174 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:15.789065 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-mz9vm"] Jun 02 21:34:15.791781 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:34:15.791739 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81fe2e5d_3e81_4d24_b43e_52ab19e21e86.slice/crio-44ef8a3bde2a17d1fb658199ac5ffa297e8bf37a6918c6bb45c4e72ee8f743d0 WatchSource:0}: Error finding container 44ef8a3bde2a17d1fb658199ac5ffa297e8bf37a6918c6bb45c4e72ee8f743d0: Status 404 returned error can't find the container with id 44ef8a3bde2a17d1fb658199ac5ffa297e8bf37a6918c6bb45c4e72ee8f743d0 Jun 02 21:34:16.347117 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:16.347076 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mz9vm" event={"ID":"81fe2e5d-3e81-4d24-b43e-52ab19e21e86","Type":"ContainerStarted","Data":"44ef8a3bde2a17d1fb658199ac5ffa297e8bf37a6918c6bb45c4e72ee8f743d0"} Jun 02 21:34:16.350394 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:16.350362 2570 generic.go:358] "Generic (PLEG): container finished" podID="291f68dc-768f-463f-8177-f9b25147390e" containerID="d99a18a52a5736ecc796d803fe922975c9a7adba297a2edac61ee25be635906d" exitCode=0 Jun 02 21:34:16.350541 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:16.350474 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerDied","Data":"d99a18a52a5736ecc796d803fe922975c9a7adba297a2edac61ee25be635906d"} Jun 02 21:34:17.356242 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:17.356206 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" event={"ID":"291f68dc-768f-463f-8177-f9b25147390e","Type":"ContainerStarted","Data":"a68c8d9ce9342b235ccab2de6751822c8d2ab1982de0c5f1d38715cd83227e12"} Jun 02 21:34:17.377473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:17.377374 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-2dk4v" podStartSLOduration=4.431005473 podStartE2EDuration="46.377357337s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:33:32.376222229 +0000 UTC m=+1.802782695" lastFinishedPulling="2026-06-02 21:34:14.322574079 +0000 UTC m=+43.749134559" observedRunningTime="2026-06-02 21:34:17.377344302 +0000 UTC m=+46.803904791" watchObservedRunningTime="2026-06-02 21:34:17.377357337 +0000 UTC m=+46.803917857" Jun 02 21:34:18.359563 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:18.359519 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerStarted","Data":"b7633315a7d53b31e0d9e5d8bb08d188e7c196c23207898f30cb1cc0de17489d"} Jun 02 21:34:18.359563 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:18.359564 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerStarted","Data":"b459e55364b411e891334c392ac9057b1cf70ce4a87c5eaeab85652a66db0257"} Jun 02 21:34:18.375635 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:18.375585 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" podStartSLOduration=18.189298882 podStartE2EDuration="31.375570263s" podCreationTimestamp="2026-06-02 21:33:47 +0000 UTC" firstStartedPulling="2026-06-02 21:34:04.539571972 +0000 UTC m=+33.966132437" lastFinishedPulling="2026-06-02 21:34:17.725843349 +0000 UTC m=+47.152403818" observedRunningTime="2026-06-02 21:34:18.3751238 +0000 UTC m=+47.801684289" watchObservedRunningTime="2026-06-02 21:34:18.375570263 +0000 UTC m=+47.802130745" Jun 02 21:34:19.842248 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:19.842211 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:19.842276 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:19.842313 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842350 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842412 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.842398313 +0000 UTC m=+65.268958779 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842414 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842416 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842478 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842464 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.842451936 +0000 UTC m=+65.269012405 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:19.842576 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.842529 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.84251698 +0000 UTC m=+65.269077445 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:19.942717 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:19.942696 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:19.942808 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.942785 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:19.942856 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:19.942826 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:34:35.942814435 +0000 UTC m=+65.369374921 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:20.365788 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:20.365754 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-mz9vm" event={"ID":"81fe2e5d-3e81-4d24-b43e-52ab19e21e86","Type":"ContainerStarted","Data":"43dbe7a24ec840cd073ae3d44c9d1d4de0c57c7e3c8c865569fe8e2c4fd96bb2"} Jun 02 21:34:20.379895 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:20.379851 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-mz9vm" podStartSLOduration=33.019021819 podStartE2EDuration="37.379839565s" podCreationTimestamp="2026-06-02 21:33:43 +0000 UTC" firstStartedPulling="2026-06-02 21:34:15.793733812 +0000 UTC m=+45.220294280" lastFinishedPulling="2026-06-02 21:34:20.154551542 +0000 UTC m=+49.581112026" observedRunningTime="2026-06-02 21:34:20.379032468 +0000 UTC m=+49.805592956" watchObservedRunningTime="2026-06-02 21:34:20.379839565 +0000 UTC m=+49.806400053" Jun 02 21:34:30.303844 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:30.303816 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-2ltrg" Jun 02 21:34:35.859258 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.859225 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:34:35.859258 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.859261 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.859281 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.859313 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859357 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859391 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859419 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:35:07.859398227 +0000 UTC m=+97.285958693 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859389 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859433 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859434 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:35:07.859428274 +0000 UTC m=+97.285988739 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:34:35.859697 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.859464 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:35:07.859454274 +0000 UTC m=+97.286014740 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:34:35.861823 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.861807 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\"" Jun 02 21:34:35.870123 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.870110 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:34:35.870165 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.870157 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:35:39.870145386 +0000 UTC m=+129.296705852 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : secret "metrics-daemon-secret" not found Jun 02 21:34:35.960285 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.960260 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:34:35.960364 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.960307 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:35.960399 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.960386 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:34:35.960430 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:34:35.960425 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:35:07.960412194 +0000 UTC m=+97.386972676 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:34:35.962779 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.962764 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\"" Jun 02 21:34:35.972555 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.972541 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\"" Jun 02 21:34:35.983657 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:35.983629 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-clgrn\" (UniqueName: \"kubernetes.io/projected/2f994d2c-9570-4f84-88b2-53c4d15e5cda-kube-api-access-clgrn\") pod \"network-check-target-scmcl\" (UID: \"2f994d2c-9570-4f84-88b2-53c4d15e5cda\") " pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:36.247776 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:36.247757 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-txpc4\"" Jun 02 21:34:36.255605 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:36.255589 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:36.372652 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:36.372621 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-scmcl"] Jun 02 21:34:36.375366 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:34:36.375332 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f994d2c_9570_4f84_88b2_53c4d15e5cda.slice/crio-955e224f02d3463672950c6fb6012f4c91688d3dc989733aff6ccae9cff71607 WatchSource:0}: Error finding container 955e224f02d3463672950c6fb6012f4c91688d3dc989733aff6ccae9cff71607: Status 404 returned error can't find the container with id 955e224f02d3463672950c6fb6012f4c91688d3dc989733aff6ccae9cff71607 Jun 02 21:34:36.405494 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:36.405469 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-scmcl" event={"ID":"2f994d2c-9570-4f84-88b2-53c4d15e5cda","Type":"ContainerStarted","Data":"955e224f02d3463672950c6fb6012f4c91688d3dc989733aff6ccae9cff71607"} Jun 02 21:34:41.417449 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:41.417417 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-scmcl" event={"ID":"2f994d2c-9570-4f84-88b2-53c4d15e5cda","Type":"ContainerStarted","Data":"7e1093f8540fb432c586c2881c4d6fbbad4b7e43046ab5e51006a9459bc41093"} Jun 02 21:34:41.417866 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:41.417535 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:34:41.431818 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:34:41.431771 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-scmcl" podStartSLOduration=65.987264736 podStartE2EDuration="1m10.431757944s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:34:36.377669791 +0000 UTC m=+65.804230271" lastFinishedPulling="2026-06-02 21:34:40.822162999 +0000 UTC m=+70.248723479" observedRunningTime="2026-06-02 21:34:41.431451102 +0000 UTC m=+70.858011591" watchObservedRunningTime="2026-06-02 21:34:41.431757944 +0000 UTC m=+70.858318433" Jun 02 21:35:07.879443 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:07.879407 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:07.879459 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879534 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879538 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879592 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:36:11.879579272 +0000 UTC m=+161.306139737 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879609 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:36:11.879600264 +0000 UTC m=+161.306160729 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:07.879642 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879739 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879751 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:35:07.879845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.879791 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:11.879780656 +0000 UTC m=+161.306341125 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:35:07.979908 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:07.979878 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:35:07.979985 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.979974 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:35:07.980032 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:07.980020 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:36:11.98000891 +0000 UTC m=+161.406569376 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:35:12.421566 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:12.421534 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-scmcl" Jun 02 21:35:39.895568 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:39.895532 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:35:39.896022 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:39.895655 2570 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found Jun 02 21:35:39.896022 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:35:39.895712 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs podName:a88961b4-8e81-4ea7-a1df-6bb7c5534afa nodeName:}" failed. No retries permitted until 2026-06-02 21:37:41.89569484 +0000 UTC m=+251.322255326 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs") pod "network-metrics-daemon-vq7rr" (UID: "a88961b4-8e81-4ea7-a1df-6bb7c5534afa") : secret "metrics-daemon-secret" not found Jun 02 21:35:55.471602 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:55.471569 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bsngs_cdf0cb33-6a7d-4973-8c72-6292c5710ef9/dns-node-resolver/0.log" Jun 02 21:35:56.274576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:35:56.274554 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7t9s9_9afa5d28-b5ca-4780-8f04-89166c9b1054/node-ca/0.log" Jun 02 21:36:07.007182 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.007126 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" podUID="9aa85708-a571-4b8c-862b-f77475fb5a2a" Jun 02 21:36:07.013256 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.013231 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" podUID="13bc0555-d53a-40ab-b925-2d7c13b26b88" Jun 02 21:36:07.046570 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.046537 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pnh9b" podUID="994dfd6d-3332-447f-869d-f4353d792e2c" Jun 02 21:36:07.051667 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.051645 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-gzfbq" podUID="68ef55e0-fb04-4198-b200-179452490e9f" Jun 02 21:36:07.126169 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.126146 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jcfwd"] Jun 02 21:36:07.128056 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.128029 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.130261 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.130244 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-stqmt\"" Jun 02 21:36:07.130353 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.130304 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\"" Jun 02 21:36:07.130353 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.130318 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\"" Jun 02 21:36:07.131027 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.131010 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\"" Jun 02 21:36:07.131104 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.131055 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\"" Jun 02 21:36:07.139812 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.139792 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jcfwd"] Jun 02 21:36:07.185721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.185690 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-data-volume\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.185887 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.185729 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.185887 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.185815 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.185887 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.185857 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-crio-socket\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.185887 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.185878 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cfkzp\" (UniqueName: \"kubernetes.io/projected/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-api-access-cfkzp\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.286829 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.286743 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-data-volume\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.286829 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.286790 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287021 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.286853 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287021 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.286898 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-crio-socket\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287021 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.286928 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cfkzp\" (UniqueName: \"kubernetes.io/projected/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-api-access-cfkzp\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287189 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.287026 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-crio-socket\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287189 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.287058 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.287189 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.287133 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-data-volume\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.287189 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.287145 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls podName:5ad3af50-c39e-4e8d-8b79-f38ba30aaae4 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:07.787122135 +0000 UTC m=+157.213682614 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jcfwd" (UID: "5ad3af50-c39e-4e8d-8b79-f38ba30aaae4") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.287382 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.287289 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.294764 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.294745 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cfkzp\" (UniqueName: \"kubernetes.io/projected/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-kube-api-access-cfkzp\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.609819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.609732 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:36:07.609819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.609756 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:36:07.609819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.609763 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:36:07.609819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.609764 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:36:07.791767 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:07.791732 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:07.791945 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.791869 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:07.791945 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:07.791940 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls podName:5ad3af50-c39e-4e8d-8b79-f38ba30aaae4 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:08.791925173 +0000 UTC m=+158.218485640 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jcfwd" (UID: "5ad3af50-c39e-4e8d-8b79-f38ba30aaae4") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:08.170482 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:08.170444 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-vq7rr" podUID="a88961b4-8e81-4ea7-a1df-6bb7c5534afa" Jun 02 21:36:08.798559 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:08.798525 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:08.798765 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:08.798712 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:08.798845 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:08.798808 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls podName:5ad3af50-c39e-4e8d-8b79-f38ba30aaae4 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:10.798786257 +0000 UTC m=+160.225346725 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jcfwd" (UID: "5ad3af50-c39e-4e8d-8b79-f38ba30aaae4") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:10.815287 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:10.815213 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:10.815657 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:10.815331 2570 secret.go:189] Couldn't get secret openshift-insights/insights-runtime-extractor-tls: secret "insights-runtime-extractor-tls" not found Jun 02 21:36:10.815657 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:10.815395 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls podName:5ad3af50-c39e-4e8d-8b79-f38ba30aaae4 nodeName:}" failed. No retries permitted until 2026-06-02 21:36:14.815382083 +0000 UTC m=+164.241942550 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "insights-runtime-extractor-tls" (UniqueName: "kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls") pod "insights-runtime-extractor-jcfwd" (UID: "5ad3af50-c39e-4e8d-8b79-f38ba30aaae4") : secret "insights-runtime-extractor-tls" not found Jun 02 21:36:11.920881 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:11.920849 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:11.920899 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") pod \"image-registry-9c855867d-6fdx9\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:11.920933 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.920994 2570 secret.go:189] Couldn't get secret openshift-network-console/networking-console-plugin-cert: secret "networking-console-plugin-cert" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921010 2570 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921041 2570 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921077 2570 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-9c855867d-6fdx9: secret "image-registry-tls" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921099 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert podName:9aa85708-a571-4b8c-862b-f77475fb5a2a nodeName:}" failed. No retries permitted until 2026-06-02 21:38:13.921081325 +0000 UTC m=+283.347641794 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "networking-console-plugin-cert" (UniqueName: "kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert") pod "networking-console-plugin-76c9bb5fdf-k5r9k" (UID: "9aa85708-a571-4b8c-862b-f77475fb5a2a") : secret "networking-console-plugin-cert" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921115 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls podName:994dfd6d-3332-447f-869d-f4353d792e2c nodeName:}" failed. No retries permitted until 2026-06-02 21:38:13.921108749 +0000 UTC m=+283.347669214 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls") pod "dns-default-pnh9b" (UID: "994dfd6d-3332-447f-869d-f4353d792e2c") : secret "dns-default-metrics-tls" not found Jun 02 21:36:11.921402 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:11.921125 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls podName:13bc0555-d53a-40ab-b925-2d7c13b26b88 nodeName:}" failed. No retries permitted until 2026-06-02 21:38:13.921120469 +0000 UTC m=+283.347680935 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls") pod "image-registry-9c855867d-6fdx9" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88") : secret "image-registry-tls" not found Jun 02 21:36:12.021805 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:12.021782 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:36:12.021963 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:12.021914 2570 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found Jun 02 21:36:12.021963 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:12.021960 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert podName:68ef55e0-fb04-4198-b200-179452490e9f nodeName:}" failed. No retries permitted until 2026-06-02 21:38:14.02194752 +0000 UTC m=+283.448507993 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert") pod "ingress-canary-gzfbq" (UID: "68ef55e0-fb04-4198-b200-179452490e9f") : secret "canary-serving-cert" not found Jun 02 21:36:14.627838 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.627812 2570 generic.go:358] "Generic (PLEG): container finished" podID="e6d2b8be-500a-44f6-9ce7-9a031012335b" containerID="59702d8136a46b0de84b00d943e65a09e95d2e74080cf9ffc8b2f84174f394bf" exitCode=255 Jun 02 21:36:14.628176 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.627890 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" event={"ID":"e6d2b8be-500a-44f6-9ce7-9a031012335b","Type":"ContainerDied","Data":"59702d8136a46b0de84b00d943e65a09e95d2e74080cf9ffc8b2f84174f394bf"} Jun 02 21:36:14.628241 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.628208 2570 scope.go:117] "RemoveContainer" containerID="59702d8136a46b0de84b00d943e65a09e95d2e74080cf9ffc8b2f84174f394bf" Jun 02 21:36:14.629227 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.629203 2570 generic.go:358] "Generic (PLEG): container finished" podID="ac7a3a47-73a4-443c-b727-2cdfd747d117" containerID="154445b44395bf41d38814df4cb62058f820a997fd5bbf7b757812111cbc7cf3" exitCode=1 Jun 02 21:36:14.629323 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.629239 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" event={"ID":"ac7a3a47-73a4-443c-b727-2cdfd747d117","Type":"ContainerDied","Data":"154445b44395bf41d38814df4cb62058f820a997fd5bbf7b757812111cbc7cf3"} Jun 02 21:36:14.629509 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.629496 2570 scope.go:117] "RemoveContainer" containerID="154445b44395bf41d38814df4cb62058f820a997fd5bbf7b757812111cbc7cf3" Jun 02 21:36:14.846640 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.846564 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:14.848813 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.848786 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/5ad3af50-c39e-4e8d-8b79-f38ba30aaae4-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jcfwd\" (UID: \"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4\") " pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:14.937526 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:14.937502 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jcfwd" Jun 02 21:36:15.052906 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.052880 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jcfwd"] Jun 02 21:36:15.055988 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:36:15.055966 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ad3af50_c39e_4e8d_8b79_f38ba30aaae4.slice/crio-7780125f5a3e847e39c12ec2dd245d4a19219749621a50024248bdfd026e92fa WatchSource:0}: Error finding container 7780125f5a3e847e39c12ec2dd245d4a19219749621a50024248bdfd026e92fa: Status 404 returned error can't find the container with id 7780125f5a3e847e39c12ec2dd245d4a19219749621a50024248bdfd026e92fa Jun 02 21:36:15.343400 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.343375 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:36:15.632817 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.632791 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-74bc9b695b-5kl4c" event={"ID":"e6d2b8be-500a-44f6-9ce7-9a031012335b","Type":"ContainerStarted","Data":"336ac8a32ef76cc23421420845f932c50e006992c414c25034c7b892d440890a"} Jun 02 21:36:15.634344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.634322 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcfwd" event={"ID":"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4","Type":"ContainerStarted","Data":"8d3cb475ba210560339de5f6d90ffad3bd5db9c0d0e87730b31c47427209ca76"} Jun 02 21:36:15.634474 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.634351 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcfwd" event={"ID":"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4","Type":"ContainerStarted","Data":"7780125f5a3e847e39c12ec2dd245d4a19219749621a50024248bdfd026e92fa"} Jun 02 21:36:15.636029 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.636002 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" event={"ID":"ac7a3a47-73a4-443c-b727-2cdfd747d117","Type":"ContainerStarted","Data":"ac2feead507165917855abe0988f7e609d6197a1801c09a5b7d71e7548191e76"} Jun 02 21:36:15.636265 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.636247 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:36:15.637700 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:15.637678 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="open-cluster-management-agent-addon/klusterlet-addon-workmgr-597d74ff77-x2qmf" Jun 02 21:36:16.639697 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:16.639656 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcfwd" event={"ID":"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4","Type":"ContainerStarted","Data":"8f3cce74a357ff53537e4f17f0f99ec5858e8b46a671db5a49de861113182b54"} Jun 02 21:36:17.643595 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:17.643565 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jcfwd" event={"ID":"5ad3af50-c39e-4e8d-8b79-f38ba30aaae4","Type":"ContainerStarted","Data":"ab26256c69535ed660fae993faf87cc5901357759c6217004533e9992ce5a324"} Jun 02 21:36:17.661694 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:17.661629 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jcfwd" podStartSLOduration=8.367701109 podStartE2EDuration="10.66161537s" podCreationTimestamp="2026-06-02 21:36:07 +0000 UTC" firstStartedPulling="2026-06-02 21:36:15.110064697 +0000 UTC m=+164.536625163" lastFinishedPulling="2026-06-02 21:36:17.403978955 +0000 UTC m=+166.830539424" observedRunningTime="2026-06-02 21:36:17.660195492 +0000 UTC m=+167.086755990" watchObservedRunningTime="2026-06-02 21:36:17.66161537 +0000 UTC m=+167.088175863" Jun 02 21:36:22.137023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:22.136983 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:36:35.375486 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.375441 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-4klbf"] Jun 02 21:36:35.383326 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.383297 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.385683 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.385661 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\"" Jun 02 21:36:35.386567 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386540 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-htktv\"" Jun 02 21:36:35.386651 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386593 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\"" Jun 02 21:36:35.386651 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386616 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\"" Jun 02 21:36:35.386772 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386663 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\"" Jun 02 21:36:35.386891 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386876 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\"" Jun 02 21:36:35.386957 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.386906 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\"" Jun 02 21:36:35.499529 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499508 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-sys\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499537 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499561 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-root\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499601 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499589 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqtvn\" (UniqueName: \"kubernetes.io/projected/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-kube-api-access-nqtvn\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499710 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499650 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499710 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499687 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499771 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499748 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-textfile\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499805 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499787 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-wtmp\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.499837 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.499823 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-metrics-client-ca\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600233 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600210 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nqtvn\" (UniqueName: \"kubernetes.io/projected/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-kube-api-access-nqtvn\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600313 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600243 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600313 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600263 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600422 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600405 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-textfile\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600431 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-wtmp\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600455 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-metrics-client-ca\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600481 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-sys\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600522 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600507 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600532 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-root\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600604 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-root\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600605 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-sys\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600605 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-wtmp\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600853 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:35.600690 2570 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found Jun 02 21:36:35.600853 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600743 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-textfile\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600853 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:35.600752 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls podName:c1238f0d-d5bd-478e-9b46-0bd7f1d575fb nodeName:}" failed. No retries permitted until 2026-06-02 21:36:36.100736267 +0000 UTC m=+185.527296748 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls") pod "node-exporter-4klbf" (UID: "c1238f0d-d5bd-478e-9b46-0bd7f1d575fb") : secret "node-exporter-tls" not found Jun 02 21:36:35.600853 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600843 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-accelerators-collector-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.600980 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.600919 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-metrics-client-ca\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.602682 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.602662 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:35.614113 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:35.614090 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqtvn\" (UniqueName: \"kubernetes.io/projected/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-kube-api-access-nqtvn\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:36.105107 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:36.105030 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:36.107299 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:36.107280 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/c1238f0d-d5bd-478e-9b46-0bd7f1d575fb-node-exporter-tls\") pod \"node-exporter-4klbf\" (UID: \"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb\") " pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:36.292073 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:36.292027 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-4klbf" Jun 02 21:36:36.300653 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:36:36.300620 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc1238f0d_d5bd_478e_9b46_0bd7f1d575fb.slice/crio-ca6f83744fc73b8646acf59e062775737c2ca146fbefeb4af9f156272af619c6 WatchSource:0}: Error finding container ca6f83744fc73b8646acf59e062775737c2ca146fbefeb4af9f156272af619c6: Status 404 returned error can't find the container with id ca6f83744fc73b8646acf59e062775737c2ca146fbefeb4af9f156272af619c6 Jun 02 21:36:36.691473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:36.691425 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4klbf" event={"ID":"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb","Type":"ContainerStarted","Data":"ca6f83744fc73b8646acf59e062775737c2ca146fbefeb4af9f156272af619c6"} Jun 02 21:36:37.695068 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:37.694971 2570 generic.go:358] "Generic (PLEG): container finished" podID="c1238f0d-d5bd-478e-9b46-0bd7f1d575fb" containerID="b8c8e9df3e9ff26dd1599116f1356aca463f41058afa584a6400ae4fb1c02068" exitCode=0 Jun 02 21:36:37.695068 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:37.695040 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4klbf" event={"ID":"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb","Type":"ContainerDied","Data":"b8c8e9df3e9ff26dd1599116f1356aca463f41058afa584a6400ae4fb1c02068"} Jun 02 21:36:38.699596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:38.699562 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4klbf" event={"ID":"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb","Type":"ContainerStarted","Data":"cbc2ea156ee7b90ecab543889093f06a5ee4149645696830c5866d32df96ad5b"} Jun 02 21:36:38.699596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:38.699596 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-4klbf" event={"ID":"c1238f0d-d5bd-478e-9b46-0bd7f1d575fb","Type":"ContainerStarted","Data":"74acbefe4ccade1c69a5f8aa61054771687f36be9683c8569f135e026953be63"} Jun 02 21:36:38.720458 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:38.720411 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-4klbf" podStartSLOduration=2.747551509 podStartE2EDuration="3.720398859s" podCreationTimestamp="2026-06-02 21:36:35 +0000 UTC" firstStartedPulling="2026-06-02 21:36:36.302402254 +0000 UTC m=+185.728962720" lastFinishedPulling="2026-06-02 21:36:37.275249587 +0000 UTC m=+186.701810070" observedRunningTime="2026-06-02 21:36:38.719468772 +0000 UTC m=+188.146029259" watchObservedRunningTime="2026-06-02 21:36:38.720398859 +0000 UTC m=+188.146959346" Jun 02 21:36:48.619177 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.619140 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9c855867d-6fdx9"] Jun 02 21:36:48.619646 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:36:48.619290 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[registry-tls], unattached volumes=[], failed to process volumes=[]: context canceled" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" podUID="13bc0555-d53a-40ab-b925-2d7c13b26b88" Jun 02 21:36:48.723116 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.723088 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:36:48.727123 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.727106 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:36:48.899576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899502 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899562 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899600 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899649 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899688 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899710 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jshc2\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899745 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca\") pod \"13bc0555-d53a-40ab-b925-2d7c13b26b88\" (UID: \"13bc0555-d53a-40ab-b925-2d7c13b26b88\") " Jun 02 21:36:48.899996 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899930 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue "" Jun 02 21:36:48.900069 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.899998 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:36:48.900254 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.900231 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:36:48.901732 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.901696 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:36:48.901824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.901733 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:36:48.901824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.901712 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:36:48.901893 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:48.901864 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2" (OuterVolumeSpecName: "kube-api-access-jshc2") pod "13bc0555-d53a-40ab-b925-2d7c13b26b88" (UID: "13bc0555-d53a-40ab-b925-2d7c13b26b88"). InnerVolumeSpecName "kube-api-access-jshc2". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:36:49.000473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000436 2570 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-image-registry-private-configuration\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000472 2570 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-bound-sa-token\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000482 2570 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/13bc0555-d53a-40ab-b925-2d7c13b26b88-installation-pull-secrets\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000639 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000491 2570 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-certificates\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000639 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000500 2570 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/13bc0555-d53a-40ab-b925-2d7c13b26b88-ca-trust-extracted\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000639 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000509 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jshc2\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-kube-api-access-jshc2\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.000639 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.000518 2570 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/13bc0555-d53a-40ab-b925-2d7c13b26b88-trusted-ca\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:49.725201 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.725172 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-9c855867d-6fdx9" Jun 02 21:36:49.769160 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.769132 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-9c855867d-6fdx9"] Jun 02 21:36:49.779815 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.779790 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-9c855867d-6fdx9"] Jun 02 21:36:49.907002 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:49.906969 2570 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/13bc0555-d53a-40ab-b925-2d7c13b26b88-registry-tls\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:36:51.139901 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:36:51.139866 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="13bc0555-d53a-40ab-b925-2d7c13b26b88" path="/var/lib/kubelet/pods/13bc0555-d53a-40ab-b925-2d7c13b26b88/volumes" Jun 02 21:37:04.341019 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:04.340985 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" podUID="c3dc1183-d347-459e-8443-709b14b46269" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:37:14.345122 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:14.345082 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" podUID="c3dc1183-d347-459e-8443-709b14b46269" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:37:24.341391 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.341353 2570 prober.go:120] "Probe failed" probeType="Liveness" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" podUID="c3dc1183-d347-459e-8443-709b14b46269" containerName="service-proxy" probeResult="failure" output="HTTP probe failed with statuscode: 500" Jun 02 21:37:24.341898 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.341429 2570 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" Jun 02 21:37:24.341941 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.341895 2570 kuberuntime_manager.go:1107] "Message for Container of pod" containerName="service-proxy" containerStatusID={"Type":"cri-o","ID":"b7633315a7d53b31e0d9e5d8bb08d188e7c196c23207898f30cb1cc0de17489d"} pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" containerMessage="Container service-proxy failed liveness probe, will be restarted" Jun 02 21:37:24.341975 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.341954 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" podUID="c3dc1183-d347-459e-8443-709b14b46269" containerName="service-proxy" containerID="cri-o://b7633315a7d53b31e0d9e5d8bb08d188e7c196c23207898f30cb1cc0de17489d" gracePeriod=30 Jun 02 21:37:24.807702 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.807669 2570 generic.go:358] "Generic (PLEG): container finished" podID="c3dc1183-d347-459e-8443-709b14b46269" containerID="b7633315a7d53b31e0d9e5d8bb08d188e7c196c23207898f30cb1cc0de17489d" exitCode=2 Jun 02 21:37:24.807866 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.807709 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerDied","Data":"b7633315a7d53b31e0d9e5d8bb08d188e7c196c23207898f30cb1cc0de17489d"} Jun 02 21:37:24.807866 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:24.807731 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/cluster-proxy-proxy-agent-78c7bd458-vkdx7" event={"ID":"c3dc1183-d347-459e-8443-709b14b46269","Type":"ContainerStarted","Data":"5227fa8912a17c594ebdc4b0d6fb1ea3f065742f995d4f271e0ea5c013076540"} Jun 02 21:37:41.982298 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:41.982209 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:37:41.984443 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:41.984421 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/a88961b4-8e81-4ea7-a1df-6bb7c5534afa-metrics-certs\") pod \"network-metrics-daemon-vq7rr\" (UID: \"a88961b4-8e81-4ea7-a1df-6bb7c5534afa\") " pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:37:42.240846 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:42.240768 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-4s2f5\"" Jun 02 21:37:42.248702 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:42.248676 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-vq7rr" Jun 02 21:37:42.367545 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:42.367517 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-vq7rr"] Jun 02 21:37:42.370932 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:37:42.370905 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda88961b4_8e81_4ea7_a1df_6bb7c5534afa.slice/crio-ce7f7fc081cca2cf758d761383a310d2fe5c1d7358e1a646a66690e27e011dd9 WatchSource:0}: Error finding container ce7f7fc081cca2cf758d761383a310d2fe5c1d7358e1a646a66690e27e011dd9: Status 404 returned error can't find the container with id ce7f7fc081cca2cf758d761383a310d2fe5c1d7358e1a646a66690e27e011dd9 Jun 02 21:37:42.848574 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:42.848543 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vq7rr" event={"ID":"a88961b4-8e81-4ea7-a1df-6bb7c5534afa","Type":"ContainerStarted","Data":"ce7f7fc081cca2cf758d761383a310d2fe5c1d7358e1a646a66690e27e011dd9"} Jun 02 21:37:43.852819 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:43.852791 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vq7rr" event={"ID":"a88961b4-8e81-4ea7-a1df-6bb7c5534afa","Type":"ContainerStarted","Data":"90f2b0e88a141df34e8d5a71ac9cf0ec9a3145c2c9fd291ffdb047ab23caf966"} Jun 02 21:37:43.853102 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:43.852824 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-vq7rr" event={"ID":"a88961b4-8e81-4ea7-a1df-6bb7c5534afa","Type":"ContainerStarted","Data":"b2238d3cbb1ed520a231e8f544e3ddf5861db609f1f78b88d22c03a599396f29"} Jun 02 21:37:43.868018 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:37:43.867969 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-vq7rr" podStartSLOduration=251.581459709 podStartE2EDuration="4m12.867956126s" podCreationTimestamp="2026-06-02 21:33:31 +0000 UTC" firstStartedPulling="2026-06-02 21:37:42.372570352 +0000 UTC m=+251.799130830" lastFinishedPulling="2026-06-02 21:37:43.659066772 +0000 UTC m=+253.085627247" observedRunningTime="2026-06-02 21:37:43.866601441 +0000 UTC m=+253.293161929" watchObservedRunningTime="2026-06-02 21:37:43.867956126 +0000 UTC m=+253.294516613" Jun 02 21:38:10.610710 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:38:10.610662 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[networking-console-plugin-cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" podUID="9aa85708-a571-4b8c-862b-f77475fb5a2a" Jun 02 21:38:10.610710 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:38:10.610662 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-gzfbq" podUID="68ef55e0-fb04-4198-b200-179452490e9f" Jun 02 21:38:10.610710 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:38:10.610662 2570 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-pnh9b" podUID="994dfd6d-3332-447f-869d-f4353d792e2c" Jun 02 21:38:10.917143 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:10.917063 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:10.917285 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:10.917063 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:38:10.917285 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:10.917063 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:38:14.008130 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.008101 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:14.008668 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.008164 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:38:14.010323 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.010293 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/994dfd6d-3332-447f-869d-f4353d792e2c-metrics-tls\") pod \"dns-default-pnh9b\" (UID: \"994dfd6d-3332-447f-869d-f4353d792e2c\") " pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:14.010459 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.010437 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/9aa85708-a571-4b8c-862b-f77475fb5a2a-networking-console-plugin-cert\") pod \"networking-console-plugin-76c9bb5fdf-k5r9k\" (UID: \"9aa85708-a571-4b8c-862b-f77475fb5a2a\") " pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:38:14.109120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.109097 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:38:14.110990 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.110975 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/68ef55e0-fb04-4198-b200-179452490e9f-cert\") pod \"ingress-canary-gzfbq\" (UID: \"68ef55e0-fb04-4198-b200-179452490e9f\") " pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:38:14.221396 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.221366 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-2xlld\"" Jun 02 21:38:14.221506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.221369 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-gs96v\"" Jun 02 21:38:14.221506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.221430 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-2lz6l\"" Jun 02 21:38:14.228115 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.228088 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-gzfbq" Jun 02 21:38:14.228205 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.228088 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:14.228259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.228097 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" Jun 02 21:38:14.367314 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.367285 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k"] Jun 02 21:38:14.370273 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:38:14.370240 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9aa85708_a571_4b8c_862b_f77475fb5a2a.slice/crio-df704cb782356fad5fbfddd215de14c841410916fb8ced68bd463ba8d27f7480 WatchSource:0}: Error finding container df704cb782356fad5fbfddd215de14c841410916fb8ced68bd463ba8d27f7480: Status 404 returned error can't find the container with id df704cb782356fad5fbfddd215de14c841410916fb8ced68bd463ba8d27f7480 Jun 02 21:38:14.590722 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.590700 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-gzfbq"] Jun 02 21:38:14.593740 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:38:14.593707 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod68ef55e0_fb04_4198_b200_179452490e9f.slice/crio-db10b4c8924771b07d7e090ea4864245ac36ff6349359b05b4c31dc1d086883c WatchSource:0}: Error finding container db10b4c8924771b07d7e090ea4864245ac36ff6349359b05b4c31dc1d086883c: Status 404 returned error can't find the container with id db10b4c8924771b07d7e090ea4864245ac36ff6349359b05b4c31dc1d086883c Jun 02 21:38:14.594084 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.594063 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-pnh9b"] Jun 02 21:38:14.596280 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:38:14.596252 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod994dfd6d_3332_447f_869d_f4353d792e2c.slice/crio-252c6d4fa025cddb7f6e261eb272c1a3aea5866646149412c8fe87651bb9b7c1 WatchSource:0}: Error finding container 252c6d4fa025cddb7f6e261eb272c1a3aea5866646149412c8fe87651bb9b7c1: Status 404 returned error can't find the container with id 252c6d4fa025cddb7f6e261eb272c1a3aea5866646149412c8fe87651bb9b7c1 Jun 02 21:38:14.928524 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.928465 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnh9b" event={"ID":"994dfd6d-3332-447f-869d-f4353d792e2c","Type":"ContainerStarted","Data":"252c6d4fa025cddb7f6e261eb272c1a3aea5866646149412c8fe87651bb9b7c1"} Jun 02 21:38:14.929411 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.929387 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" event={"ID":"9aa85708-a571-4b8c-862b-f77475fb5a2a","Type":"ContainerStarted","Data":"df704cb782356fad5fbfddd215de14c841410916fb8ced68bd463ba8d27f7480"} Jun 02 21:38:14.930310 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:14.930290 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gzfbq" event={"ID":"68ef55e0-fb04-4198-b200-179452490e9f","Type":"ContainerStarted","Data":"db10b4c8924771b07d7e090ea4864245ac36ff6349359b05b4c31dc1d086883c"} Jun 02 21:38:16.939657 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:16.939614 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" event={"ID":"9aa85708-a571-4b8c-862b-f77475fb5a2a","Type":"ContainerStarted","Data":"a61c8b1e55d41f61ffacbca95be6edc6aab455e658a6d8667fe90dd6905ec28c"} Jun 02 21:38:16.942292 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:16.942251 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-gzfbq" event={"ID":"68ef55e0-fb04-4198-b200-179452490e9f","Type":"ContainerStarted","Data":"5883403b5efacbe11fcc8627dbf316551e7e0d26246e0179af6cd96e60ad416f"} Jun 02 21:38:16.944009 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:16.943980 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnh9b" event={"ID":"994dfd6d-3332-447f-869d-f4353d792e2c","Type":"ContainerStarted","Data":"01b2051ef6ec06cca59615e8bc5e24466118da8ce2c8cef06ad2fce4f72467e1"} Jun 02 21:38:16.962698 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:16.962648 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-76c9bb5fdf-k5r9k" podStartSLOduration=271.558002849 podStartE2EDuration="4m33.962632395s" podCreationTimestamp="2026-06-02 21:33:43 +0000 UTC" firstStartedPulling="2026-06-02 21:38:14.372525427 +0000 UTC m=+283.799085892" lastFinishedPulling="2026-06-02 21:38:16.777154963 +0000 UTC m=+286.203715438" observedRunningTime="2026-06-02 21:38:16.960565091 +0000 UTC m=+286.387125579" watchObservedRunningTime="2026-06-02 21:38:16.962632395 +0000 UTC m=+286.389192884" Jun 02 21:38:17.947879 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:17.947841 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-pnh9b" event={"ID":"994dfd6d-3332-447f-869d-f4353d792e2c","Type":"ContainerStarted","Data":"1c054e5a67a562d2631d0b09c35075ddb9839a0576b3ca54a7c702f7bc67c1b5"} Jun 02 21:38:17.969307 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:17.969266 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-pnh9b" podStartSLOduration=252.789602456 podStartE2EDuration="4m14.969252748s" podCreationTimestamp="2026-06-02 21:34:03 +0000 UTC" firstStartedPulling="2026-06-02 21:38:14.597918627 +0000 UTC m=+284.024479106" lastFinishedPulling="2026-06-02 21:38:16.777568926 +0000 UTC m=+286.204129398" observedRunningTime="2026-06-02 21:38:17.967733694 +0000 UTC m=+287.394294194" watchObservedRunningTime="2026-06-02 21:38:17.969252748 +0000 UTC m=+287.395813270" Jun 02 21:38:17.969598 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:17.969574 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-gzfbq" podStartSLOduration=252.783638873 podStartE2EDuration="4m14.969567857s" podCreationTimestamp="2026-06-02 21:34:03 +0000 UTC" firstStartedPulling="2026-06-02 21:38:14.595793967 +0000 UTC m=+284.022354449" lastFinishedPulling="2026-06-02 21:38:16.781722967 +0000 UTC m=+286.208283433" observedRunningTime="2026-06-02 21:38:16.974322845 +0000 UTC m=+286.400883336" watchObservedRunningTime="2026-06-02 21:38:17.969567857 +0000 UTC m=+287.396128345" Jun 02 21:38:18.950344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:18.950317 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:28.955251 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:28.955217 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-pnh9b" Jun 02 21:38:31.055634 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:31.055603 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:38:31.056066 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:31.055610 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:38:31.060234 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:38:31.060219 2570 kubelet.go:1628] "Image garbage collection succeeded" Jun 02 21:41:03.729762 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.729687 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5"] Jun 02 21:41:03.732544 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.732528 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.734804 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.734784 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"openshift-lws-operator-dockercfg-gw778\"" Jun 02 21:41:03.734804 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.734797 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\"" Jun 02 21:41:03.735503 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.735478 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\"" Jun 02 21:41:03.740473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.740452 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5"] Jun 02 21:41:03.796742 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.796719 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4482w\" (UniqueName: \"kubernetes.io/projected/c89ff155-20e1-42b9-b4ed-7aa065ebd812-kube-api-access-4482w\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.796832 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.796755 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c89ff155-20e1-42b9-b4ed-7aa065ebd812-tmp\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.897693 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.897670 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4482w\" (UniqueName: \"kubernetes.io/projected/c89ff155-20e1-42b9-b4ed-7aa065ebd812-kube-api-access-4482w\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.897780 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.897703 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c89ff155-20e1-42b9-b4ed-7aa065ebd812-tmp\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.897979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.897964 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c89ff155-20e1-42b9-b4ed-7aa065ebd812-tmp\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:03.905129 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:03.905102 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4482w\" (UniqueName: \"kubernetes.io/projected/c89ff155-20e1-42b9-b4ed-7aa065ebd812-kube-api-access-4482w\") pod \"openshift-lws-operator-bfc7f696d-p67z5\" (UID: \"c89ff155-20e1-42b9-b4ed-7aa065ebd812\") " pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:04.041843 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:04.041790 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" Jun 02 21:41:04.159842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:04.159769 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5"] Jun 02 21:41:04.162632 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:41:04.162605 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc89ff155_20e1_42b9_b4ed_7aa065ebd812.slice/crio-b0ccb96f91f9da9ea67b1b7057afac4a71535a2b146c5995042b51f201f36c24 WatchSource:0}: Error finding container b0ccb96f91f9da9ea67b1b7057afac4a71535a2b146c5995042b51f201f36c24: Status 404 returned error can't find the container with id b0ccb96f91f9da9ea67b1b7057afac4a71535a2b146c5995042b51f201f36c24 Jun 02 21:41:04.164427 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:04.164408 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:41:04.340717 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:04.340648 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" event={"ID":"c89ff155-20e1-42b9-b4ed-7aa065ebd812","Type":"ContainerStarted","Data":"b0ccb96f91f9da9ea67b1b7057afac4a71535a2b146c5995042b51f201f36c24"} Jun 02 21:41:09.355074 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:09.355025 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" event={"ID":"c89ff155-20e1-42b9-b4ed-7aa065ebd812","Type":"ContainerStarted","Data":"ec067c9fb9dc4c85c758e8a824abd72762ca42285715f324a9afc7c8b3eb7a74"} Jun 02 21:41:09.370638 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:09.370595 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/openshift-lws-operator-bfc7f696d-p67z5" podStartSLOduration=1.250190945 podStartE2EDuration="6.370582299s" podCreationTimestamp="2026-06-02 21:41:03 +0000 UTC" firstStartedPulling="2026-06-02 21:41:04.164566768 +0000 UTC m=+453.591127248" lastFinishedPulling="2026-06-02 21:41:09.284958121 +0000 UTC m=+458.711518602" observedRunningTime="2026-06-02 21:41:09.369336986 +0000 UTC m=+458.795897486" watchObservedRunningTime="2026-06-02 21:41:09.370582299 +0000 UTC m=+458.797142786" Jun 02 21:41:24.659455 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.659416 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv"] Jun 02 21:41:24.662444 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.662424 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.665463 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.665439 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\"" Jun 02 21:41:24.665596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.665439 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\"" Jun 02 21:41:24.665596 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.665472 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\"" Jun 02 21:41:24.665751 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.665711 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\"" Jun 02 21:41:24.665950 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.665934 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-77t8f\"" Jun 02 21:41:24.678520 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.678496 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv"] Jun 02 21:41:24.733110 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.733077 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.733255 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.733128 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v274q\" (UniqueName: \"kubernetes.io/projected/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-kube-api-access-v274q\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.733255 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.733179 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.733255 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.733200 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.733255 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.733218 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834014 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.833980 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v274q\" (UniqueName: \"kubernetes.io/projected/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-kube-api-access-v274q\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834014 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.834021 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834261 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.834057 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834261 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.834076 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834261 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.834106 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.834442 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.834424 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manifests\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-manifests\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.836379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.836352 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-tmp\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.836662 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.836645 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-webhook-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.836714 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.836695 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-apiservice-cert\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.843174 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.843153 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v274q\" (UniqueName: \"kubernetes.io/projected/0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1-kube-api-access-v274q\") pod \"opendatahub-operator-controller-manager-8b78d96d9-sgfrv\" (UID: \"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1\") " pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:24.972366 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:24.972340 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:25.095384 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:25.095346 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv"] Jun 02 21:41:25.098489 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:41:25.098465 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0e532d1d_4a4d_4eb8_870c_4d4f72ea02c1.slice/crio-fda0b50a2cc29d88966d70afae47bcd206dd3f71c4668fe337c7e90c31509899 WatchSource:0}: Error finding container fda0b50a2cc29d88966d70afae47bcd206dd3f71c4668fe337c7e90c31509899: Status 404 returned error can't find the container with id fda0b50a2cc29d88966d70afae47bcd206dd3f71c4668fe337c7e90c31509899 Jun 02 21:41:25.393448 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:25.393366 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" event={"ID":"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1","Type":"ContainerStarted","Data":"fda0b50a2cc29d88966d70afae47bcd206dd3f71c4668fe337c7e90c31509899"} Jun 02 21:41:28.407160 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:28.407128 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" event={"ID":"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1","Type":"ContainerStarted","Data":"7512df8a5c7e86d49a9a01442c63e556a459d266b7563a4dc239ef6e10c126d7"} Jun 02 21:41:29.411084 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:29.411006 2570 generic.go:358] "Generic (PLEG): container finished" podID="0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1" containerID="7512df8a5c7e86d49a9a01442c63e556a459d266b7563a4dc239ef6e10c126d7" exitCode=0 Jun 02 21:41:29.411394 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:29.411083 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" event={"ID":"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1","Type":"ContainerDied","Data":"7512df8a5c7e86d49a9a01442c63e556a459d266b7563a4dc239ef6e10c126d7"} Jun 02 21:41:30.415435 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:30.415405 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" event={"ID":"0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1","Type":"ContainerStarted","Data":"5f48e05a8b7dda9b873232c2c207992676be38a35f8f9602732495023ff80bc1"} Jun 02 21:41:30.415796 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:30.415545 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:30.437646 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:30.437595 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" podStartSLOduration=1.807052064 podStartE2EDuration="6.437579804s" podCreationTimestamp="2026-06-02 21:41:24 +0000 UTC" firstStartedPulling="2026-06-02 21:41:25.100741028 +0000 UTC m=+474.527301493" lastFinishedPulling="2026-06-02 21:41:29.731268764 +0000 UTC m=+479.157829233" observedRunningTime="2026-06-02 21:41:30.437187902 +0000 UTC m=+479.863748390" watchObservedRunningTime="2026-06-02 21:41:30.437579804 +0000 UTC m=+479.864140294" Jun 02 21:41:41.420950 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:41.420917 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-8b78d96d9-sgfrv" Jun 02 21:41:44.335061 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.335011 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m"] Jun 02 21:41:44.341262 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.341232 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.343566 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.343547 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-creds\"" Jun 02 21:41:44.344425 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.344401 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\"" Jun 02 21:41:44.344533 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.344422 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-dockercfg-98lv6\"" Jun 02 21:41:44.344533 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.344414 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"kube-auth-proxy-tls\"" Jun 02 21:41:44.344533 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.344499 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\"" Jun 02 21:41:44.352685 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.352665 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m"] Jun 02 21:41:44.429222 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.429183 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/odh-model-controller-76588b95-s5g6q"] Jun 02 21:41:44.431179 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.431164 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.433443 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.433418 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-webhook-cert\"" Jun 02 21:41:44.433570 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.433502 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"odh-model-controller-dockercfg-l4kgf\"" Jun 02 21:41:44.440316 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.440292 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-s5g6q"] Jun 02 21:41:44.474264 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.474231 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-55v5r\" (UniqueName: \"kubernetes.io/projected/7c618835-b03f-49ab-a49c-c23c2d710d45-kube-api-access-55v5r\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.474264 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.474268 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d338093-f371-4091-b33f-b7c26b4baf05-tmp\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.474473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.474289 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6v86j\" (UniqueName: \"kubernetes.io/projected/0d338093-f371-4091-b33f-b7c26b4baf05-kube-api-access-6v86j\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.474473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.474382 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.474473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.474450 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d338093-f371-4091-b33f-b7c26b4baf05-tls-certs\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.575597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.575560 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d338093-f371-4091-b33f-b7c26b4baf05-tls-certs\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.575771 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.575610 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-55v5r\" (UniqueName: \"kubernetes.io/projected/7c618835-b03f-49ab-a49c-c23c2d710d45-kube-api-access-55v5r\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.575771 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.575630 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d338093-f371-4091-b33f-b7c26b4baf05-tmp\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.575771 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.575657 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6v86j\" (UniqueName: \"kubernetes.io/projected/0d338093-f371-4091-b33f-b7c26b4baf05-kube-api-access-6v86j\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.575771 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.575706 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.575989 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:41:44.575789 2570 secret.go:189] Couldn't get secret opendatahub/odh-model-controller-webhook-cert: secret "odh-model-controller-webhook-cert" not found Jun 02 21:41:44.575989 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:41:44.575863 2570 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert podName:7c618835-b03f-49ab-a49c-c23c2d710d45 nodeName:}" failed. No retries permitted until 2026-06-02 21:41:45.075844241 +0000 UTC m=+494.502404722 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert") pod "odh-model-controller-76588b95-s5g6q" (UID: "7c618835-b03f-49ab-a49c-c23c2d710d45") : secret "odh-model-controller-webhook-cert" not found Jun 02 21:41:44.578163 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.578135 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/0d338093-f371-4091-b33f-b7c26b4baf05-tmp\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.578402 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.578382 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/0d338093-f371-4091-b33f-b7c26b4baf05-tls-certs\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.593027 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.592961 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-55v5r\" (UniqueName: \"kubernetes.io/projected/7c618835-b03f-49ab-a49c-c23c2d710d45-kube-api-access-55v5r\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:44.595018 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.595001 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6v86j\" (UniqueName: \"kubernetes.io/projected/0d338093-f371-4091-b33f-b7c26b4baf05-kube-api-access-6v86j\") pod \"kube-auth-proxy-dcdbb48fb-jbf5m\" (UID: \"0d338093-f371-4091-b33f-b7c26b4baf05\") " pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.651773 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.651736 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" Jun 02 21:41:44.772583 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:44.772560 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m"] Jun 02 21:41:44.775167 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:41:44.775135 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0d338093_f371_4091_b33f_b7c26b4baf05.slice/crio-339ed04d89a7f5f8961943476ff51a65ef2c22b9470d925bb00707ff94b58865 WatchSource:0}: Error finding container 339ed04d89a7f5f8961943476ff51a65ef2c22b9470d925bb00707ff94b58865: Status 404 returned error can't find the container with id 339ed04d89a7f5f8961943476ff51a65ef2c22b9470d925bb00707ff94b58865 Jun 02 21:41:45.079435 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.079398 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:45.081751 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.081731 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/7c618835-b03f-49ab-a49c-c23c2d710d45-cert\") pod \"odh-model-controller-76588b95-s5g6q\" (UID: \"7c618835-b03f-49ab-a49c-c23c2d710d45\") " pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:45.224664 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.224635 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp"] Jun 02 21:41:45.227056 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.227023 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.231502 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.231475 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\"" Jun 02 21:41:45.231670 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.231477 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-68gnz\"" Jun 02 21:41:45.231670 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.231477 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\"" Jun 02 21:41:45.231670 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.231487 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\"" Jun 02 21:41:45.237683 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.237663 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp"] Jun 02 21:41:45.281769 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.281736 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1cb1052-6417-4612-b03c-c1bf98655ff6-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.281938 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.281788 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.281938 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.281846 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5xfhd\" (UniqueName: \"kubernetes.io/projected/e1cb1052-6417-4612-b03c-c1bf98655ff6-kube-api-access-5xfhd\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.281938 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.281922 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.343002 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.342913 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:45.383079 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.383029 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1cb1052-6417-4612-b03c-c1bf98655ff6-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.383206 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.383095 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.383998 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.383711 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/e1cb1052-6417-4612-b03c-c1bf98655ff6-manager-config\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.383998 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.383785 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5xfhd\" (UniqueName: \"kubernetes.io/projected/e1cb1052-6417-4612-b03c-c1bf98655ff6-kube-api-access-5xfhd\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.383998 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.383845 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.386127 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.386109 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.386462 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.386445 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/e1cb1052-6417-4612-b03c-c1bf98655ff6-metrics-cert\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.391805 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.391771 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5xfhd\" (UniqueName: \"kubernetes.io/projected/e1cb1052-6417-4612-b03c-c1bf98655ff6-kube-api-access-5xfhd\") pod \"lws-controller-manager-78c4cd8bbd-dlhgp\" (UID: \"e1cb1052-6417-4612-b03c-c1bf98655ff6\") " pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.460011 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.459529 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" event={"ID":"0d338093-f371-4091-b33f-b7c26b4baf05","Type":"ContainerStarted","Data":"339ed04d89a7f5f8961943476ff51a65ef2c22b9470d925bb00707ff94b58865"} Jun 02 21:41:45.498877 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.498771 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/odh-model-controller-76588b95-s5g6q"] Jun 02 21:41:45.503195 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:41:45.503161 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c618835_b03f_49ab_a49c_c23c2d710d45.slice/crio-6b6c99a76e40448feefd8f917cce8e2b3055ef8996e6b4f549dcf7d68b1aa301 WatchSource:0}: Error finding container 6b6c99a76e40448feefd8f917cce8e2b3055ef8996e6b4f549dcf7d68b1aa301: Status 404 returned error can't find the container with id 6b6c99a76e40448feefd8f917cce8e2b3055ef8996e6b4f549dcf7d68b1aa301 Jun 02 21:41:45.539214 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.539184 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:45.702543 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:45.702481 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp"] Jun 02 21:41:45.706142 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:41:45.706107 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1cb1052_6417_4612_b03c_c1bf98655ff6.slice/crio-0e8f36c695287eff761fa723a8171b2552b09a7a38f2e194e675fea3f2a23299 WatchSource:0}: Error finding container 0e8f36c695287eff761fa723a8171b2552b09a7a38f2e194e675fea3f2a23299: Status 404 returned error can't find the container with id 0e8f36c695287eff761fa723a8171b2552b09a7a38f2e194e675fea3f2a23299 Jun 02 21:41:46.466984 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:46.466893 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-s5g6q" event={"ID":"7c618835-b03f-49ab-a49c-c23c2d710d45","Type":"ContainerStarted","Data":"6b6c99a76e40448feefd8f917cce8e2b3055ef8996e6b4f549dcf7d68b1aa301"} Jun 02 21:41:46.470272 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:46.470209 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" event={"ID":"e1cb1052-6417-4612-b03c-c1bf98655ff6","Type":"ContainerStarted","Data":"0e8f36c695287eff761fa723a8171b2552b09a7a38f2e194e675fea3f2a23299"} Jun 02 21:41:50.484034 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.483998 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" event={"ID":"e1cb1052-6417-4612-b03c-c1bf98655ff6","Type":"ContainerStarted","Data":"a2dfa01973edbfd19e2ef63f5ee48e6adb65d64966327fd371ed993cf9ddb693"} Jun 02 21:41:50.484666 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.484079 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:41:50.485302 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.485278 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" event={"ID":"0d338093-f371-4091-b33f-b7c26b4baf05","Type":"ContainerStarted","Data":"3590276cf4d6a6303b8f5c4d0aa4e9ffde2b679772bb45b03a8928affb8ef464"} Jun 02 21:41:50.486627 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.486607 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/odh-model-controller-76588b95-s5g6q" event={"ID":"7c618835-b03f-49ab-a49c-c23c2d710d45","Type":"ContainerStarted","Data":"272ee87b5d89f670b63c3115213348d8d2e1c8a6297c8211586c644b82d1cf84"} Jun 02 21:41:50.486767 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.486741 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:41:50.499272 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.499230 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" podStartSLOduration=1.450588996 podStartE2EDuration="5.499220153s" podCreationTimestamp="2026-06-02 21:41:45 +0000 UTC" firstStartedPulling="2026-06-02 21:41:45.708700891 +0000 UTC m=+495.135261358" lastFinishedPulling="2026-06-02 21:41:49.757332028 +0000 UTC m=+499.183892515" observedRunningTime="2026-06-02 21:41:50.497991195 +0000 UTC m=+499.924551683" watchObservedRunningTime="2026-06-02 21:41:50.499220153 +0000 UTC m=+499.925780641" Jun 02 21:41:50.511897 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.511856 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/kube-auth-proxy-dcdbb48fb-jbf5m" podStartSLOduration=1.519587067 podStartE2EDuration="6.511843436s" podCreationTimestamp="2026-06-02 21:41:44 +0000 UTC" firstStartedPulling="2026-06-02 21:41:44.777097374 +0000 UTC m=+494.203657854" lastFinishedPulling="2026-06-02 21:41:49.769353741 +0000 UTC m=+499.195914223" observedRunningTime="2026-06-02 21:41:50.511261454 +0000 UTC m=+499.937821945" watchObservedRunningTime="2026-06-02 21:41:50.511843436 +0000 UTC m=+499.938403924" Jun 02 21:41:50.526166 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:41:50.526122 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/odh-model-controller-76588b95-s5g6q" podStartSLOduration=2.256716286 podStartE2EDuration="6.526108467s" podCreationTimestamp="2026-06-02 21:41:44 +0000 UTC" firstStartedPulling="2026-06-02 21:41:45.504841498 +0000 UTC m=+494.931401971" lastFinishedPulling="2026-06-02 21:41:49.774233683 +0000 UTC m=+499.200794152" observedRunningTime="2026-06-02 21:41:50.524860107 +0000 UTC m=+499.951420603" watchObservedRunningTime="2026-06-02 21:41:50.526108467 +0000 UTC m=+499.952668956" Jun 02 21:42:01.492061 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:42:01.492019 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-78c4cd8bbd-dlhgp" Jun 02 21:42:01.492454 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:42:01.492088 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/odh-model-controller-76588b95-s5g6q" Jun 02 21:43:31.074149 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:31.074120 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:43:31.074957 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:31.074938 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:43:33.895519 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.895479 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v"] Jun 02 21:43:33.898648 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.898628 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:33.902153 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.902136 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\"" Jun 02 21:43:33.902292 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.902277 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\"" Jun 02 21:43:33.907378 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.907359 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\"" Jun 02 21:43:33.907465 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.907412 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\"" Jun 02 21:43:33.907662 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.907649 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-2ng5x\"" Jun 02 21:43:33.911698 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:33.911679 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v"] Jun 02 21:43:34.085517 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.085488 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c14b9c-833b-4c4a-b50b-584d0d5570d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.085648 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.085546 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mjz62\" (UniqueName: \"kubernetes.io/projected/87c14b9c-833b-4c4a-b50b-584d0d5570d5-kube-api-access-mjz62\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.085648 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.085617 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/87c14b9c-833b-4c4a-b50b-584d0d5570d5-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.186511 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.186455 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c14b9c-833b-4c4a-b50b-584d0d5570d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.186511 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.186498 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mjz62\" (UniqueName: \"kubernetes.io/projected/87c14b9c-833b-4c4a-b50b-584d0d5570d5-kube-api-access-mjz62\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.186635 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.186524 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/87c14b9c-833b-4c4a-b50b-584d0d5570d5-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.187100 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.187075 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/87c14b9c-833b-4c4a-b50b-584d0d5570d5-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.188690 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.188672 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/87c14b9c-833b-4c4a-b50b-584d0d5570d5-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.196143 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.196123 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mjz62\" (UniqueName: \"kubernetes.io/projected/87c14b9c-833b-4c4a-b50b-584d0d5570d5-kube-api-access-mjz62\") pod \"kuadrant-console-plugin-6cb54b5c86-bkq8v\" (UID: \"87c14b9c-833b-4c4a-b50b-584d0d5570d5\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.206946 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.206928 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" Jun 02 21:43:34.334668 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.334638 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v"] Jun 02 21:43:34.337978 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:43:34.337953 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod87c14b9c_833b_4c4a_b50b_584d0d5570d5.slice/crio-c2684adb113e041ed34c36a0a2d98b8c1a802d7cc4b7def0a4a8493e252255ad WatchSource:0}: Error finding container c2684adb113e041ed34c36a0a2d98b8c1a802d7cc4b7def0a4a8493e252255ad: Status 404 returned error can't find the container with id c2684adb113e041ed34c36a0a2d98b8c1a802d7cc4b7def0a4a8493e252255ad Jun 02 21:43:34.782445 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:34.782413 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" event={"ID":"87c14b9c-833b-4c4a-b50b-584d0d5570d5","Type":"ContainerStarted","Data":"c2684adb113e041ed34c36a0a2d98b8c1a802d7cc4b7def0a4a8493e252255ad"} Jun 02 21:43:58.863132 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:58.863098 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" event={"ID":"87c14b9c-833b-4c4a-b50b-584d0d5570d5","Type":"ContainerStarted","Data":"b3bfb6c4600d0326f74bca4c45ec007d9ff49f24475c70745a8d2915aa1ae1c8"} Jun 02 21:43:58.891194 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:43:58.891136 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-bkq8v" podStartSLOduration=2.34008286 podStartE2EDuration="25.891121307s" podCreationTimestamp="2026-06-02 21:43:33 +0000 UTC" firstStartedPulling="2026-06-02 21:43:34.33965057 +0000 UTC m=+603.766211035" lastFinishedPulling="2026-06-02 21:43:57.890689013 +0000 UTC m=+627.317249482" observedRunningTime="2026-06-02 21:43:58.889882621 +0000 UTC m=+628.316443108" watchObservedRunningTime="2026-06-02 21:43:58.891121307 +0000 UTC m=+628.317681832" Jun 02 21:44:29.624112 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.624072 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:29.679320 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.679290 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:29.679500 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.679414 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:29.681737 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.681712 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-4xhjm\"" Jun 02 21:44:29.774609 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.774570 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpnk7\" (UniqueName: \"kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7\") pod \"authorino-f99f4b5cd-m9mp8\" (UID: \"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b\") " pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:29.825103 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.825065 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:44:29.828386 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.828371 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:44:29.835145 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.835092 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:44:29.875080 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.874976 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cgjd8\" (UniqueName: \"kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8\") pod \"authorino-7498df8756-7tdmm\" (UID: \"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2\") " pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:44:29.875080 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.875023 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qpnk7\" (UniqueName: \"kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7\") pod \"authorino-f99f4b5cd-m9mp8\" (UID: \"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b\") " pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:29.883342 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.883317 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpnk7\" (UniqueName: \"kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7\") pod \"authorino-f99f4b5cd-m9mp8\" (UID: \"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b\") " pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:29.976081 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.976029 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cgjd8\" (UniqueName: \"kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8\") pod \"authorino-7498df8756-7tdmm\" (UID: \"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2\") " pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:44:29.984454 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.984433 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cgjd8\" (UniqueName: \"kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8\") pod \"authorino-7498df8756-7tdmm\" (UID: \"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2\") " pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:44:29.988189 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:29.988167 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:30.114164 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:30.114131 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:30.117228 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:44:30.117187 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podff7d322d_c1f9_45a6_ae2c_b586cb9ad54b.slice/crio-43fa3d1a4062f8cee6fe94f507ae51a1f0a60e4b7d0288a1488863bcf2153735 WatchSource:0}: Error finding container 43fa3d1a4062f8cee6fe94f507ae51a1f0a60e4b7d0288a1488863bcf2153735: Status 404 returned error can't find the container with id 43fa3d1a4062f8cee6fe94f507ae51a1f0a60e4b7d0288a1488863bcf2153735 Jun 02 21:44:30.138252 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:30.138196 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:44:30.257379 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:30.257349 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:44:30.260367 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:44:30.260344 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1fc56e0a_14f5_48ed_bfb5_fb4b1d6cc8f2.slice/crio-a1a0ba620614c7a25fbc1c848d201feab2022475f59489bdc1dc77cee19c980b WatchSource:0}: Error finding container a1a0ba620614c7a25fbc1c848d201feab2022475f59489bdc1dc77cee19c980b: Status 404 returned error can't find the container with id a1a0ba620614c7a25fbc1c848d201feab2022475f59489bdc1dc77cee19c980b Jun 02 21:44:30.962182 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:30.962145 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7tdmm" event={"ID":"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2","Type":"ContainerStarted","Data":"a1a0ba620614c7a25fbc1c848d201feab2022475f59489bdc1dc77cee19c980b"} Jun 02 21:44:30.965643 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:30.965579 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" event={"ID":"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b","Type":"ContainerStarted","Data":"43fa3d1a4062f8cee6fe94f507ae51a1f0a60e4b7d0288a1488863bcf2153735"} Jun 02 21:44:33.977066 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:33.977017 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7tdmm" event={"ID":"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2","Type":"ContainerStarted","Data":"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b"} Jun 02 21:44:33.978334 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:33.978304 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" event={"ID":"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b","Type":"ContainerStarted","Data":"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786"} Jun 02 21:44:33.993754 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:33.993674 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-7tdmm" podStartSLOduration=1.5061955889999998 podStartE2EDuration="4.99366319s" podCreationTimestamp="2026-06-02 21:44:29 +0000 UTC" firstStartedPulling="2026-06-02 21:44:30.261755328 +0000 UTC m=+659.688315794" lastFinishedPulling="2026-06-02 21:44:33.749222923 +0000 UTC m=+663.175783395" observedRunningTime="2026-06-02 21:44:33.991808611 +0000 UTC m=+663.418369111" watchObservedRunningTime="2026-06-02 21:44:33.99366319 +0000 UTC m=+663.420223679" Jun 02 21:44:34.007229 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:34.007171 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" podStartSLOduration=1.389776326 podStartE2EDuration="5.007150819s" podCreationTimestamp="2026-06-02 21:44:29 +0000 UTC" firstStartedPulling="2026-06-02 21:44:30.118452689 +0000 UTC m=+659.545013156" lastFinishedPulling="2026-06-02 21:44:33.735827177 +0000 UTC m=+663.162387649" observedRunningTime="2026-06-02 21:44:34.005205756 +0000 UTC m=+663.431766241" watchObservedRunningTime="2026-06-02 21:44:34.007150819 +0000 UTC m=+663.433711307" Jun 02 21:44:34.018102 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:34.018073 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:35.984368 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:35.984331 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" podUID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" containerName="authorino" containerID="cri-o://2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786" gracePeriod=30 Jun 02 21:44:36.212300 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.212279 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:36.328473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.328416 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qpnk7\" (UniqueName: \"kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7\") pod \"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b\" (UID: \"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b\") " Jun 02 21:44:36.330289 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.330263 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7" (OuterVolumeSpecName: "kube-api-access-qpnk7") pod "ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" (UID: "ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b"). InnerVolumeSpecName "kube-api-access-qpnk7". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:44:36.429292 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.429267 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qpnk7\" (UniqueName: \"kubernetes.io/projected/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b-kube-api-access-qpnk7\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:44:36.988511 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.988483 2570 generic.go:358] "Generic (PLEG): container finished" podID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" containerID="2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786" exitCode=0 Jun 02 21:44:36.988842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.988529 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" Jun 02 21:44:36.988842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.988566 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" event={"ID":"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b","Type":"ContainerDied","Data":"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786"} Jun 02 21:44:36.988842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.988604 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-f99f4b5cd-m9mp8" event={"ID":"ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b","Type":"ContainerDied","Data":"43fa3d1a4062f8cee6fe94f507ae51a1f0a60e4b7d0288a1488863bcf2153735"} Jun 02 21:44:36.988842 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.988624 2570 scope.go:117] "RemoveContainer" containerID="2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786" Jun 02 21:44:36.996554 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.996537 2570 scope.go:117] "RemoveContainer" containerID="2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786" Jun 02 21:44:36.996790 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:44:36.996769 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786\": container with ID starting with 2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786 not found: ID does not exist" containerID="2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786" Jun 02 21:44:36.996837 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:36.996799 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786"} err="failed to get container status \"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786\": rpc error: code = NotFound desc = could not find container \"2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786\": container with ID starting with 2ee81572cc9521f5c68d9e59b13fa80f62504f1e3a14070666d1419c7d734786 not found: ID does not exist" Jun 02 21:44:37.007125 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:37.007101 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:37.010459 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:37.010440 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-f99f4b5cd-m9mp8"] Jun 02 21:44:37.141157 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:44:37.141136 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" path="/var/lib/kubelet/pods/ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b/volumes" Jun 02 21:45:29.282681 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.282643 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:45:29.283230 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.283027 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" containerName="authorino" Jun 02 21:45:29.283230 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.283066 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" containerName="authorino" Jun 02 21:45:29.283230 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.283148 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="ff7d322d-c1f9-45a6-ae2c-b586cb9ad54b" containerName="authorino" Jun 02 21:45:29.284880 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.284858 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:29.287128 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.287109 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:45:29.287229 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.287169 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:45:29.287229 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.287219 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-56rz9\"" Jun 02 21:45:29.287924 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.287909 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:45:29.294023 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.294002 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:45:29.388591 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.388556 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dp22w\" (UniqueName: \"kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w\") pod \"maas-keycloak-0\" (UID: \"f6614ff9-917b-4c6d-b45c-3b78c0e760dc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:29.489006 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.488967 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dp22w\" (UniqueName: \"kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w\") pod \"maas-keycloak-0\" (UID: \"f6614ff9-917b-4c6d-b45c-3b78c0e760dc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:29.497725 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.497695 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dp22w\" (UniqueName: \"kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w\") pod \"maas-keycloak-0\" (UID: \"f6614ff9-917b-4c6d-b45c-3b78c0e760dc\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:29.596088 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.595992 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:29.720293 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:29.720269 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:45:29.722661 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:45:29.722633 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf6614ff9_917b_4c6d_b45c_3b78c0e760dc.slice/crio-f6e7889600ea36785ad247c5939abe3e40b0fb9f9b0440fb8395fda733cd33e3 WatchSource:0}: Error finding container f6e7889600ea36785ad247c5939abe3e40b0fb9f9b0440fb8395fda733cd33e3: Status 404 returned error can't find the container with id f6e7889600ea36785ad247c5939abe3e40b0fb9f9b0440fb8395fda733cd33e3 Jun 02 21:45:30.149016 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:30.148984 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6614ff9-917b-4c6d-b45c-3b78c0e760dc","Type":"ContainerStarted","Data":"f6e7889600ea36785ad247c5939abe3e40b0fb9f9b0440fb8395fda733cd33e3"} Jun 02 21:45:34.561138 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:34.561116 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:45:35.172914 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:35.172880 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6614ff9-917b-4c6d-b45c-3b78c0e760dc","Type":"ContainerStarted","Data":"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5"} Jun 02 21:45:35.191217 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:35.191172 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.3564404890000001 podStartE2EDuration="6.191159174s" podCreationTimestamp="2026-06-02 21:45:29 +0000 UTC" firstStartedPulling="2026-06-02 21:45:29.724026544 +0000 UTC m=+719.150587022" lastFinishedPulling="2026-06-02 21:45:34.558745236 +0000 UTC m=+723.985305707" observedRunningTime="2026-06-02 21:45:35.189142561 +0000 UTC m=+724.615703049" watchObservedRunningTime="2026-06-02 21:45:35.191159174 +0000 UTC m=+724.617719662" Jun 02 21:45:35.596442 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:35.596398 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:35.598174 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:35.598139 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:36.596710 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:36.596674 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:37.596956 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:37.596903 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:38.597131 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:38.597077 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:39.596409 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:39.596361 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:39.596840 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:39.596787 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:40.596863 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:40.596798 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:41.596998 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:41.596938 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:42.596473 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:42.596432 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:43.597223 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:43.597178 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:44.597273 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:44.597224 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:45.596779 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:45.596734 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:46.597300 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:46.597251 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.22:9000/health/started\": dial tcp 10.132.0.22:9000: connect: connection refused" Jun 02 21:45:47.725341 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:47.725295 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:47.742787 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:47.742739 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:45:57.732156 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:57.732121 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:45:58.617790 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.617754 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:45:58.624225 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.624205 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:45:58.626294 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.626275 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:45:58.725404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.725379 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wxxr4\" (UniqueName: \"kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4\") pod \"authorino-8b475cf9f-jqf7b\" (UID: \"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580\") " pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:45:58.826670 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.826643 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wxxr4\" (UniqueName: \"kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4\") pod \"authorino-8b475cf9f-jqf7b\" (UID: \"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580\") " pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:45:58.833991 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.833965 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wxxr4\" (UniqueName: \"kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4\") pod \"authorino-8b475cf9f-jqf7b\" (UID: \"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580\") " pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:45:58.860577 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.860555 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:45:58.860701 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.860690 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:45:58.886813 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.886788 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:45:58.889649 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.889627 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:45:58.894979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.894954 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:45:58.927967 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.927943 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-n9zvn\" (UniqueName: \"kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn\") pod \"authorino-6f48988f-78g9k\" (UID: \"1512f578-9289-475f-976b-8ee4808230db\") " pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:45:58.981804 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:58.981680 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:45:58.984453 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:45:58.984430 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod714ca2f2_0d40_4d8a_accd_d9f2f4dbf580.slice/crio-51559ac5759110fe549f4c6aea9280ccfde9d867c37886fc9fe4da4f6d5ab484 WatchSource:0}: Error finding container 51559ac5759110fe549f4c6aea9280ccfde9d867c37886fc9fe4da4f6d5ab484: Status 404 returned error can't find the container with id 51559ac5759110fe549f4c6aea9280ccfde9d867c37886fc9fe4da4f6d5ab484 Jun 02 21:45:59.028588 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:59.028567 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-n9zvn\" (UniqueName: \"kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn\") pod \"authorino-6f48988f-78g9k\" (UID: \"1512f578-9289-475f-976b-8ee4808230db\") " pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:45:59.035391 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:59.035365 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-n9zvn\" (UniqueName: \"kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn\") pod \"authorino-6f48988f-78g9k\" (UID: \"1512f578-9289-475f-976b-8ee4808230db\") " pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:45:59.200360 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:59.200311 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:45:59.258030 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:59.258003 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" event={"ID":"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580","Type":"ContainerStarted","Data":"51559ac5759110fe549f4c6aea9280ccfde9d867c37886fc9fe4da4f6d5ab484"} Jun 02 21:45:59.317780 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:45:59.317741 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:45:59.320560 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:45:59.320533 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod1512f578_9289_475f_976b_8ee4808230db.slice/crio-d0550d6dae62186fc4e356939f182602864f9b8bdbf6779e262c7f86d7e48d75 WatchSource:0}: Error finding container d0550d6dae62186fc4e356939f182602864f9b8bdbf6779e262c7f86d7e48d75: Status 404 returned error can't find the container with id d0550d6dae62186fc4e356939f182602864f9b8bdbf6779e262c7f86d7e48d75 Jun 02 21:46:00.261890 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.261857 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" event={"ID":"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580","Type":"ContainerStarted","Data":"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5"} Jun 02 21:46:00.262321 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.261902 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" podUID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" containerName="authorino" containerID="cri-o://0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5" gracePeriod=30 Jun 02 21:46:00.263267 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.263244 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f48988f-78g9k" event={"ID":"1512f578-9289-475f-976b-8ee4808230db","Type":"ContainerStarted","Data":"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c"} Jun 02 21:46:00.263385 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.263277 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f48988f-78g9k" event={"ID":"1512f578-9289-475f-976b-8ee4808230db","Type":"ContainerStarted","Data":"d0550d6dae62186fc4e356939f182602864f9b8bdbf6779e262c7f86d7e48d75"} Jun 02 21:46:00.275613 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.275568 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" podStartSLOduration=1.912749043 podStartE2EDuration="2.275555109s" podCreationTimestamp="2026-06-02 21:45:58 +0000 UTC" firstStartedPulling="2026-06-02 21:45:58.985597788 +0000 UTC m=+748.412158267" lastFinishedPulling="2026-06-02 21:45:59.348403863 +0000 UTC m=+748.774964333" observedRunningTime="2026-06-02 21:46:00.275347758 +0000 UTC m=+749.701908242" watchObservedRunningTime="2026-06-02 21:46:00.275555109 +0000 UTC m=+749.702115597" Jun 02 21:46:00.287519 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.287484 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-6f48988f-78g9k" podStartSLOduration=1.860796642 podStartE2EDuration="2.287473015s" podCreationTimestamp="2026-06-02 21:45:58 +0000 UTC" firstStartedPulling="2026-06-02 21:45:59.321859721 +0000 UTC m=+748.748420191" lastFinishedPulling="2026-06-02 21:45:59.748536097 +0000 UTC m=+749.175096564" observedRunningTime="2026-06-02 21:46:00.286748545 +0000 UTC m=+749.713309034" watchObservedRunningTime="2026-06-02 21:46:00.287473015 +0000 UTC m=+749.714033502" Jun 02 21:46:00.308520 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.308496 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:46:00.308702 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.308682 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-7tdmm" podUID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" containerName="authorino" containerID="cri-o://8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b" gracePeriod=30 Jun 02 21:46:00.500793 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.500768 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:46:00.504423 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.503836 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.509687 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.506506 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-server-cert\"" Jun 02 21:46:00.513480 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.513426 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:46:00.542822 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.542801 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.542955 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.542936 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tdsvd\" (UniqueName: \"kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.551323 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.551307 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:46:00.567821 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.567804 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:46:00.643597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.643576 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cgjd8\" (UniqueName: \"kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8\") pod \"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2\" (UID: \"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2\") " Jun 02 21:46:00.643709 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.643627 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wxxr4\" (UniqueName: \"kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4\") pod \"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580\" (UID: \"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580\") " Jun 02 21:46:00.643757 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.643737 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tdsvd\" (UniqueName: \"kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.643806 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.643790 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.645654 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.645622 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4" (OuterVolumeSpecName: "kube-api-access-wxxr4") pod "714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" (UID: "714ca2f2-0d40-4d8a-accd-d9f2f4dbf580"). InnerVolumeSpecName "kube-api-access-wxxr4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:46:00.645737 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.645625 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8" (OuterVolumeSpecName: "kube-api-access-cgjd8") pod "1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" (UID: "1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2"). InnerVolumeSpecName "kube-api-access-cgjd8". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:46:00.645957 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.645941 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.651214 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.651193 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tdsvd\" (UniqueName: \"kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd\") pod \"authorino-8cf67d99c-ms876\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.744488 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.744464 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cgjd8\" (UniqueName: \"kubernetes.io/projected/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2-kube-api-access-cgjd8\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:46:00.744488 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.744487 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wxxr4\" (UniqueName: \"kubernetes.io/projected/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580-kube-api-access-wxxr4\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:46:00.822107 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.822059 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:46:00.943404 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:00.943370 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:46:00.945658 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:46:00.945630 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod38846959_50a3_4b32_a2c4_5e295b77e74c.slice/crio-6fe5a1d8cfb98792a7edaf80f7ef6dbd5f21e41abeb8aad0e2c5979b847a6bb8 WatchSource:0}: Error finding container 6fe5a1d8cfb98792a7edaf80f7ef6dbd5f21e41abeb8aad0e2c5979b847a6bb8: Status 404 returned error can't find the container with id 6fe5a1d8cfb98792a7edaf80f7ef6dbd5f21e41abeb8aad0e2c5979b847a6bb8 Jun 02 21:46:01.267688 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.267661 2570 generic.go:358] "Generic (PLEG): container finished" podID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" containerID="0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5" exitCode=0 Jun 02 21:46:01.268076 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.267717 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" Jun 02 21:46:01.268076 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.267753 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" event={"ID":"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580","Type":"ContainerDied","Data":"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5"} Jun 02 21:46:01.268076 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.267798 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8b475cf9f-jqf7b" event={"ID":"714ca2f2-0d40-4d8a-accd-d9f2f4dbf580","Type":"ContainerDied","Data":"51559ac5759110fe549f4c6aea9280ccfde9d867c37886fc9fe4da4f6d5ab484"} Jun 02 21:46:01.268076 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.267819 2570 scope.go:117] "RemoveContainer" containerID="0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5" Jun 02 21:46:01.269013 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.268856 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8cf67d99c-ms876" event={"ID":"38846959-50a3-4b32-a2c4-5e295b77e74c","Type":"ContainerStarted","Data":"6fe5a1d8cfb98792a7edaf80f7ef6dbd5f21e41abeb8aad0e2c5979b847a6bb8"} Jun 02 21:46:01.270009 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.269988 2570 generic.go:358] "Generic (PLEG): container finished" podID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" containerID="8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b" exitCode=0 Jun 02 21:46:01.270137 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.270029 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-7tdmm" Jun 02 21:46:01.270137 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.270076 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7tdmm" event={"ID":"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2","Type":"ContainerDied","Data":"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b"} Jun 02 21:46:01.270137 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.270104 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-7tdmm" event={"ID":"1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2","Type":"ContainerDied","Data":"a1a0ba620614c7a25fbc1c848d201feab2022475f59489bdc1dc77cee19c980b"} Jun 02 21:46:01.288638 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.288608 2570 scope.go:117] "RemoveContainer" containerID="0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5" Jun 02 21:46:01.288890 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:46:01.288868 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5\": container with ID starting with 0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5 not found: ID does not exist" containerID="0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5" Jun 02 21:46:01.288942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.288900 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5"} err="failed to get container status \"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5\": rpc error: code = NotFound desc = could not find container \"0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5\": container with ID starting with 0b919071bb7dcef7a8c9c53f609baafb76f611c25c7574a4f300ed0af5eec3f5 not found: ID does not exist" Jun 02 21:46:01.288942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.288917 2570 scope.go:117] "RemoveContainer" containerID="8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b" Jun 02 21:46:01.306137 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.306117 2570 scope.go:117] "RemoveContainer" containerID="8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b" Jun 02 21:46:01.306427 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:46:01.306398 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b\": container with ID starting with 8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b not found: ID does not exist" containerID="8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b" Jun 02 21:46:01.306491 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.306438 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b"} err="failed to get container status \"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b\": rpc error: code = NotFound desc = could not find container \"8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b\": container with ID starting with 8645e0d8a985638fab9a4c9b5dde977e029204b1d2d9598f3a358ea83edfd75b not found: ID does not exist" Jun 02 21:46:01.314628 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.314606 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:46:01.317171 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.317150 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-7tdmm"] Jun 02 21:46:01.325646 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.325626 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:46:01.329712 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:01.329693 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8b475cf9f-jqf7b"] Jun 02 21:46:02.274153 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.274119 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8cf67d99c-ms876" event={"ID":"38846959-50a3-4b32-a2c4-5e295b77e74c","Type":"ContainerStarted","Data":"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810"} Jun 02 21:46:02.288348 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.288309 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-8cf67d99c-ms876" podStartSLOduration=1.97254132 podStartE2EDuration="2.288295089s" podCreationTimestamp="2026-06-02 21:46:00 +0000 UTC" firstStartedPulling="2026-06-02 21:46:00.94687589 +0000 UTC m=+750.373436370" lastFinishedPulling="2026-06-02 21:46:01.262629667 +0000 UTC m=+750.689190139" observedRunningTime="2026-06-02 21:46:02.287403886 +0000 UTC m=+751.713964373" watchObservedRunningTime="2026-06-02 21:46:02.288295089 +0000 UTC m=+751.714855579" Jun 02 21:46:02.310443 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.310419 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:46:02.310626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.310605 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-6f48988f-78g9k" podUID="1512f578-9289-475f-976b-8ee4808230db" containerName="authorino" containerID="cri-o://8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c" gracePeriod=30 Jun 02 21:46:02.551505 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.551484 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:46:02.660138 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.660110 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-n9zvn\" (UniqueName: \"kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn\") pod \"1512f578-9289-475f-976b-8ee4808230db\" (UID: \"1512f578-9289-475f-976b-8ee4808230db\") " Jun 02 21:46:02.661979 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.661960 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn" (OuterVolumeSpecName: "kube-api-access-n9zvn") pod "1512f578-9289-475f-976b-8ee4808230db" (UID: "1512f578-9289-475f-976b-8ee4808230db"). InnerVolumeSpecName "kube-api-access-n9zvn". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:46:02.761292 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:02.761268 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-n9zvn\" (UniqueName: \"kubernetes.io/projected/1512f578-9289-475f-976b-8ee4808230db-kube-api-access-n9zvn\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:46:03.141485 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.141458 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" path="/var/lib/kubelet/pods/1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2/volumes" Jun 02 21:46:03.141747 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.141735 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" path="/var/lib/kubelet/pods/714ca2f2-0d40-4d8a-accd-d9f2f4dbf580/volumes" Jun 02 21:46:03.279560 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.279527 2570 generic.go:358] "Generic (PLEG): container finished" podID="1512f578-9289-475f-976b-8ee4808230db" containerID="8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c" exitCode=0 Jun 02 21:46:03.279884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.279583 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-6f48988f-78g9k" Jun 02 21:46:03.279884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.279618 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f48988f-78g9k" event={"ID":"1512f578-9289-475f-976b-8ee4808230db","Type":"ContainerDied","Data":"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c"} Jun 02 21:46:03.279884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.279651 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-6f48988f-78g9k" event={"ID":"1512f578-9289-475f-976b-8ee4808230db","Type":"ContainerDied","Data":"d0550d6dae62186fc4e356939f182602864f9b8bdbf6779e262c7f86d7e48d75"} Jun 02 21:46:03.279884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.279666 2570 scope.go:117] "RemoveContainer" containerID="8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c" Jun 02 21:46:03.286974 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.286959 2570 scope.go:117] "RemoveContainer" containerID="8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c" Jun 02 21:46:03.287234 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:46:03.287212 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c\": container with ID starting with 8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c not found: ID does not exist" containerID="8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c" Jun 02 21:46:03.287310 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.287239 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c"} err="failed to get container status \"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c\": rpc error: code = NotFound desc = could not find container \"8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c\": container with ID starting with 8fd4d84049e1742b16c153b2e5165a5cb27d20a84e81cfc2688647416b13469c not found: ID does not exist" Jun 02 21:46:03.294179 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.294159 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:46:03.296261 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.296243 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-6f48988f-78g9k"] Jun 02 21:46:03.506433 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506409 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:03.506694 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506682 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1512f578-9289-475f-976b-8ee4808230db" containerName="authorino" Jun 02 21:46:03.506745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506696 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1512f578-9289-475f-976b-8ee4808230db" containerName="authorino" Jun 02 21:46:03.506745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506711 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" containerName="authorino" Jun 02 21:46:03.506745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506716 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" containerName="authorino" Jun 02 21:46:03.506745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506728 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" containerName="authorino" Jun 02 21:46:03.506745 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506733 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" containerName="authorino" Jun 02 21:46:03.506878 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506789 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="714ca2f2-0d40-4d8a-accd-d9f2f4dbf580" containerName="authorino" Jun 02 21:46:03.506878 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506797 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="1fc56e0a-14f5-48ed-bfb5-fb4b1d6cc8f2" containerName="authorino" Jun 02 21:46:03.506878 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.506805 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="1512f578-9289-475f-976b-8ee4808230db" containerName="authorino" Jun 02 21:46:03.510427 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.510412 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:03.512550 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.512528 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 21:46:03.512642 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.512604 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2xv5f\"" Jun 02 21:46:03.518755 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.518736 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:03.565943 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.565921 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nbh6j\" (UniqueName: \"kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j\") pod \"maas-controller-7cbcd6449d-tcvng\" (UID: \"010b7a72-828d-4788-8fe8-145417e3bf9e\") " pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:03.655838 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.655815 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:46:03.658513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.658497 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:03.665513 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.665490 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:46:03.666810 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.666788 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nbh6j\" (UniqueName: \"kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j\") pod \"maas-controller-7cbcd6449d-tcvng\" (UID: \"010b7a72-828d-4788-8fe8-145417e3bf9e\") " pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:03.673695 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.673672 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nbh6j\" (UniqueName: \"kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j\") pod \"maas-controller-7cbcd6449d-tcvng\" (UID: \"010b7a72-828d-4788-8fe8-145417e3bf9e\") " pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:03.768101 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.768041 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sglh4\" (UniqueName: \"kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4\") pod \"maas-controller-764f4b5cd5-lnlcf\" (UID: \"56001ee8-ef73-4855-bfb1-0faf8fc4218d\") " pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:03.821107 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.821087 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:03.869133 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.869105 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sglh4\" (UniqueName: \"kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4\") pod \"maas-controller-764f4b5cd5-lnlcf\" (UID: \"56001ee8-ef73-4855-bfb1-0faf8fc4218d\") " pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:03.878580 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.878554 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sglh4\" (UniqueName: \"kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4\") pod \"maas-controller-764f4b5cd5-lnlcf\" (UID: \"56001ee8-ef73-4855-bfb1-0faf8fc4218d\") " pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:03.940080 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.940033 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:03.942695 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:46:03.942668 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod010b7a72_828d_4788_8fe8_145417e3bf9e.slice/crio-f26801d22b6ffe9a488f691248d2ff9ef50de713c28598d19176b27d2a158317 WatchSource:0}: Error finding container f26801d22b6ffe9a488f691248d2ff9ef50de713c28598d19176b27d2a158317: Status 404 returned error can't find the container with id f26801d22b6ffe9a488f691248d2ff9ef50de713c28598d19176b27d2a158317 Jun 02 21:46:03.969174 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:03.969154 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:04.090077 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:04.090030 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:46:04.092619 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:46:04.092593 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod56001ee8_ef73_4855_bfb1_0faf8fc4218d.slice/crio-f6ea7f5268a108d2afa7fa5128e93f7eed1cc4592680d88856383c64ba4b6627 WatchSource:0}: Error finding container f6ea7f5268a108d2afa7fa5128e93f7eed1cc4592680d88856383c64ba4b6627: Status 404 returned error can't find the container with id f6ea7f5268a108d2afa7fa5128e93f7eed1cc4592680d88856383c64ba4b6627 Jun 02 21:46:04.285308 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:04.285280 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" event={"ID":"010b7a72-828d-4788-8fe8-145417e3bf9e","Type":"ContainerStarted","Data":"f26801d22b6ffe9a488f691248d2ff9ef50de713c28598d19176b27d2a158317"} Jun 02 21:46:04.286299 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:04.286265 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" event={"ID":"56001ee8-ef73-4855-bfb1-0faf8fc4218d","Type":"ContainerStarted","Data":"f6ea7f5268a108d2afa7fa5128e93f7eed1cc4592680d88856383c64ba4b6627"} Jun 02 21:46:05.143787 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:05.143431 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="1512f578-9289-475f-976b-8ee4808230db" path="/var/lib/kubelet/pods/1512f578-9289-475f-976b-8ee4808230db/volumes" Jun 02 21:46:07.300166 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.300135 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" event={"ID":"010b7a72-828d-4788-8fe8-145417e3bf9e","Type":"ContainerStarted","Data":"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160"} Jun 02 21:46:07.300581 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.300195 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:07.301413 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.301392 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" event={"ID":"56001ee8-ef73-4855-bfb1-0faf8fc4218d","Type":"ContainerStarted","Data":"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de"} Jun 02 21:46:07.301491 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.301430 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:07.315652 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.315608 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" podStartSLOduration=1.6959362470000001 podStartE2EDuration="4.31559609s" podCreationTimestamp="2026-06-02 21:46:03 +0000 UTC" firstStartedPulling="2026-06-02 21:46:03.943939294 +0000 UTC m=+753.370499774" lastFinishedPulling="2026-06-02 21:46:06.56359915 +0000 UTC m=+755.990159617" observedRunningTime="2026-06-02 21:46:07.313767299 +0000 UTC m=+756.740327788" watchObservedRunningTime="2026-06-02 21:46:07.31559609 +0000 UTC m=+756.742156578" Jun 02 21:46:07.329375 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:07.329337 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" podStartSLOduration=1.849293955 podStartE2EDuration="4.329325914s" podCreationTimestamp="2026-06-02 21:46:03 +0000 UTC" firstStartedPulling="2026-06-02 21:46:04.093859606 +0000 UTC m=+753.520420073" lastFinishedPulling="2026-06-02 21:46:06.573891548 +0000 UTC m=+756.000452032" observedRunningTime="2026-06-02 21:46:07.327641546 +0000 UTC m=+756.754202045" watchObservedRunningTime="2026-06-02 21:46:07.329325914 +0000 UTC m=+756.755886401" Jun 02 21:46:18.309778 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.309750 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:18.310357 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.309806 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:46:18.397368 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.397329 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:18.397517 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.397502 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" podUID="010b7a72-828d-4788-8fe8-145417e3bf9e" containerName="manager" containerID="cri-o://0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160" gracePeriod=30 Jun 02 21:46:18.632805 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.632785 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:18.785576 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.785548 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nbh6j\" (UniqueName: \"kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j\") pod \"010b7a72-828d-4788-8fe8-145417e3bf9e\" (UID: \"010b7a72-828d-4788-8fe8-145417e3bf9e\") " Jun 02 21:46:18.787603 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.787580 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j" (OuterVolumeSpecName: "kube-api-access-nbh6j") pod "010b7a72-828d-4788-8fe8-145417e3bf9e" (UID: "010b7a72-828d-4788-8fe8-145417e3bf9e"). InnerVolumeSpecName "kube-api-access-nbh6j". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:46:18.886120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:18.886066 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nbh6j\" (UniqueName: \"kubernetes.io/projected/010b7a72-828d-4788-8fe8-145417e3bf9e-kube-api-access-nbh6j\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:46:19.337710 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.337678 2570 generic.go:358] "Generic (PLEG): container finished" podID="010b7a72-828d-4788-8fe8-145417e3bf9e" containerID="0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160" exitCode=0 Jun 02 21:46:19.338029 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.337736 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" Jun 02 21:46:19.338029 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.337761 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" event={"ID":"010b7a72-828d-4788-8fe8-145417e3bf9e","Type":"ContainerDied","Data":"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160"} Jun 02 21:46:19.338029 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.337797 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cbcd6449d-tcvng" event={"ID":"010b7a72-828d-4788-8fe8-145417e3bf9e","Type":"ContainerDied","Data":"f26801d22b6ffe9a488f691248d2ff9ef50de713c28598d19176b27d2a158317"} Jun 02 21:46:19.338029 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.337814 2570 scope.go:117] "RemoveContainer" containerID="0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160" Jun 02 21:46:19.345462 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.345449 2570 scope.go:117] "RemoveContainer" containerID="0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160" Jun 02 21:46:19.345673 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:46:19.345659 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160\": container with ID starting with 0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160 not found: ID does not exist" containerID="0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160" Jun 02 21:46:19.345712 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.345680 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160"} err="failed to get container status \"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160\": rpc error: code = NotFound desc = could not find container \"0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160\": container with ID starting with 0e15ffa88378b6d9edf8a651737c3b32bd5dc636d34880fe713480f08e17a160 not found: ID does not exist" Jun 02 21:46:19.352988 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.352968 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:19.356460 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:19.356442 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cbcd6449d-tcvng"] Jun 02 21:46:21.141494 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:21.141460 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="010b7a72-828d-4788-8fe8-145417e3bf9e" path="/var/lib/kubelet/pods/010b7a72-828d-4788-8fe8-145417e3bf9e/volumes" Jun 02 21:46:36.770823 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.770793 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-6bcd9cbcc4-dss74"] Jun 02 21:46:36.771277 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.771193 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="010b7a72-828d-4788-8fe8-145417e3bf9e" containerName="manager" Jun 02 21:46:36.771277 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.771211 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="010b7a72-828d-4788-8fe8-145417e3bf9e" containerName="manager" Jun 02 21:46:36.771401 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.771287 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="010b7a72-828d-4788-8fe8-145417e3bf9e" containerName="manager" Jun 02 21:46:36.775644 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.775622 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:36.777775 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.777756 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-r2mw2\"" Jun 02 21:46:36.777884 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.777747 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\"" Jun 02 21:46:36.784927 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.784899 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6bcd9cbcc4-dss74"] Jun 02 21:46:36.922246 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.922225 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hwsq4\" (UniqueName: \"kubernetes.io/projected/9c723b92-d186-4e78-a2c1-0181d7e84dd6-kube-api-access-hwsq4\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:36.922358 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:36.922258 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c723b92-d186-4e78-a2c1-0181d7e84dd6-maas-api-tls\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.023195 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.023120 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hwsq4\" (UniqueName: \"kubernetes.io/projected/9c723b92-d186-4e78-a2c1-0181d7e84dd6-kube-api-access-hwsq4\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.023195 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.023163 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c723b92-d186-4e78-a2c1-0181d7e84dd6-maas-api-tls\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.025477 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.025453 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/9c723b92-d186-4e78-a2c1-0181d7e84dd6-maas-api-tls\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.030638 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.030617 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hwsq4\" (UniqueName: \"kubernetes.io/projected/9c723b92-d186-4e78-a2c1-0181d7e84dd6-kube-api-access-hwsq4\") pod \"maas-api-6bcd9cbcc4-dss74\" (UID: \"9c723b92-d186-4e78-a2c1-0181d7e84dd6\") " pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.085597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.085569 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:37.211649 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.211620 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-6bcd9cbcc4-dss74"] Jun 02 21:46:37.214260 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:46:37.214229 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9c723b92_d186_4e78_a2c1_0181d7e84dd6.slice/crio-2fe53b18342382a07b0e80b679d7d3481f15e1385256048c5fe0f222b943588c WatchSource:0}: Error finding container 2fe53b18342382a07b0e80b679d7d3481f15e1385256048c5fe0f222b943588c: Status 404 returned error can't find the container with id 2fe53b18342382a07b0e80b679d7d3481f15e1385256048c5fe0f222b943588c Jun 02 21:46:37.215697 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.215680 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 21:46:37.392682 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:37.392601 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" event={"ID":"9c723b92-d186-4e78-a2c1-0181d7e84dd6","Type":"ContainerStarted","Data":"2fe53b18342382a07b0e80b679d7d3481f15e1385256048c5fe0f222b943588c"} Jun 02 21:46:39.402142 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:39.402105 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" event={"ID":"9c723b92-d186-4e78-a2c1-0181d7e84dd6","Type":"ContainerStarted","Data":"6825ee55d4e9e003d43fb9019f51ae1feb8755a6b6c4aec74c64d37203f2127a"} Jun 02 21:46:39.402510 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:39.402213 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:39.422264 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:39.422219 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" podStartSLOduration=1.755711386 podStartE2EDuration="3.422208694s" podCreationTimestamp="2026-06-02 21:46:36 +0000 UTC" firstStartedPulling="2026-06-02 21:46:37.215801017 +0000 UTC m=+786.642361483" lastFinishedPulling="2026-06-02 21:46:38.88229832 +0000 UTC m=+788.308858791" observedRunningTime="2026-06-02 21:46:39.420140081 +0000 UTC m=+788.846700570" watchObservedRunningTime="2026-06-02 21:46:39.422208694 +0000 UTC m=+788.848769182" Jun 02 21:46:45.409923 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:45.409846 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-api-6bcd9cbcc4-dss74" Jun 02 21:46:47.264134 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:47.264098 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:47.264502 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:47.264324 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" containerID="cri-o://81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5" gracePeriod=30 Jun 02 21:46:47.729270 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:47.729225 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:46:49.297325 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.297303 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.308974 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.308946 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dp22w\" (UniqueName: \"kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w\") pod \"f6614ff9-917b-4c6d-b45c-3b78c0e760dc\" (UID: \"f6614ff9-917b-4c6d-b45c-3b78c0e760dc\") " Jun 02 21:46:49.311006 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.310981 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w" (OuterVolumeSpecName: "kube-api-access-dp22w") pod "f6614ff9-917b-4c6d-b45c-3b78c0e760dc" (UID: "f6614ff9-917b-4c6d-b45c-3b78c0e760dc"). InnerVolumeSpecName "kube-api-access-dp22w". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:46:49.410260 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.410227 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dp22w\" (UniqueName: \"kubernetes.io/projected/f6614ff9-917b-4c6d-b45c-3b78c0e760dc-kube-api-access-dp22w\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:46:49.433595 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.433529 2570 generic.go:358] "Generic (PLEG): container finished" podID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerID="81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5" exitCode=143 Jun 02 21:46:49.433595 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.433583 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.433723 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.433612 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6614ff9-917b-4c6d-b45c-3b78c0e760dc","Type":"ContainerDied","Data":"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5"} Jun 02 21:46:49.433723 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.433654 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"f6614ff9-917b-4c6d-b45c-3b78c0e760dc","Type":"ContainerDied","Data":"f6e7889600ea36785ad247c5939abe3e40b0fb9f9b0440fb8395fda733cd33e3"} Jun 02 21:46:49.433723 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.433669 2570 scope.go:117] "RemoveContainer" containerID="81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5" Jun 02 21:46:49.442401 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.442384 2570 scope.go:117] "RemoveContainer" containerID="81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5" Jun 02 21:46:49.442633 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:46:49.442616 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5\": container with ID starting with 81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5 not found: ID does not exist" containerID="81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5" Jun 02 21:46:49.442696 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.442640 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5"} err="failed to get container status \"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5\": rpc error: code = NotFound desc = could not find container \"81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5\": container with ID starting with 81aa153030989a55ab69f1bea8049343cd2f7c90ee70cd6ff674a54e44d91df5 not found: ID does not exist" Jun 02 21:46:49.453444 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.453420 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:49.460087 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.460067 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:49.481633 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.481609 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:49.481871 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.481859 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" Jun 02 21:46:49.481911 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.481873 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" Jun 02 21:46:49.481947 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.481916 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" containerName="keycloak" Jun 02 21:46:49.485716 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.485702 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.488157 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.488137 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"openshift-service-ca.crt\"" Jun 02 21:46:49.488157 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.488149 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"maas-keycloak-initial-admin\"" Jun 02 21:46:49.488294 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.488218 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"keycloak-test-realms\"" Jun 02 21:46:49.488294 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.488273 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"keycloak-system\"/\"kube-root-ca.crt\"" Jun 02 21:46:49.488542 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.488525 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"keycloak-system\"/\"default-dockercfg-56rz9\"" Jun 02 21:46:49.491875 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.491854 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:49.510744 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.510726 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rh2dq\" (UniqueName: \"kubernetes.io/projected/8f856529-c5c8-4d94-bb26-efd30e4c131f-kube-api-access-rh2dq\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.510837 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.510761 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8f856529-c5c8-4d94-bb26-efd30e4c131f-test-realms\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.611276 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.611258 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8f856529-c5c8-4d94-bb26-efd30e4c131f-test-realms\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.611357 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.611313 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rh2dq\" (UniqueName: \"kubernetes.io/projected/8f856529-c5c8-4d94-bb26-efd30e4c131f-kube-api-access-rh2dq\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.611870 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.611852 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"test-realms\" (UniqueName: \"kubernetes.io/configmap/8f856529-c5c8-4d94-bb26-efd30e4c131f-test-realms\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.619794 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.619771 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rh2dq\" (UniqueName: \"kubernetes.io/projected/8f856529-c5c8-4d94-bb26-efd30e4c131f-kube-api-access-rh2dq\") pod \"maas-keycloak-0\" (UID: \"8f856529-c5c8-4d94-bb26-efd30e4c131f\") " pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.795722 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.795698 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:49.913224 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:49.913198 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["keycloak-system/maas-keycloak-0"] Jun 02 21:46:49.915674 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:46:49.915635 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f856529_c5c8_4d94_bb26_efd30e4c131f.slice/crio-cc79eb9649427c6de10ec59046ec17e7083f8a5fb4f0f9cf94e1f22239f3116e WatchSource:0}: Error finding container cc79eb9649427c6de10ec59046ec17e7083f8a5fb4f0f9cf94e1f22239f3116e: Status 404 returned error can't find the container with id cc79eb9649427c6de10ec59046ec17e7083f8a5fb4f0f9cf94e1f22239f3116e Jun 02 21:46:50.438926 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:50.438895 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8f856529-c5c8-4d94-bb26-efd30e4c131f","Type":"ContainerStarted","Data":"a9983c7657aa56e780c2100d6cd48f2cb172e0f3b6786c66340812246d5f9f0f"} Jun 02 21:46:50.438926 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:50.438930 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="keycloak-system/maas-keycloak-0" event={"ID":"8f856529-c5c8-4d94-bb26-efd30e4c131f","Type":"ContainerStarted","Data":"cc79eb9649427c6de10ec59046ec17e7083f8a5fb4f0f9cf94e1f22239f3116e"} Jun 02 21:46:50.457498 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:50.457433 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="keycloak-system/maas-keycloak-0" podStartSLOduration=1.155266963 podStartE2EDuration="1.45741125s" podCreationTimestamp="2026-06-02 21:46:49 +0000 UTC" firstStartedPulling="2026-06-02 21:46:49.917079052 +0000 UTC m=+799.343639519" lastFinishedPulling="2026-06-02 21:46:50.219223327 +0000 UTC m=+799.645783806" observedRunningTime="2026-06-02 21:46:50.454362934 +0000 UTC m=+799.880923423" watchObservedRunningTime="2026-06-02 21:46:50.45741125 +0000 UTC m=+799.883971739" Jun 02 21:46:50.796309 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:50.796268 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:50.797807 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:50.797774 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:51.143329 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:51.143235 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="f6614ff9-917b-4c6d-b45c-3b78c0e760dc" path="/var/lib/kubelet/pods/f6614ff9-917b-4c6d-b45c-3b78c0e760dc/volumes" Jun 02 21:46:51.796815 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:51.796768 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:52.797063 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:52.796996 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:53.796803 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:53.796750 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:54.796823 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:54.796772 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:55.796509 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:55.796452 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:56.796781 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:56.796736 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:57.796586 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:57.796535 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:58.796687 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:58.796640 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:46:59.796418 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:59.796367 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:46:59.797472 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:46:59.797434 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:47:00.796811 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:00.796752 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:47:01.796776 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:01.796720 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:47:02.796249 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:02.796205 2570 prober.go:120] "Probe failed" probeType="Startup" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="Get \"http://10.132.0.29:9000/health/started\": dial tcp 10.132.0.29:9000: connect: connection refused" Jun 02 21:47:03.913430 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:03.913391 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="keycloak-system/maas-keycloak-0" Jun 02 21:47:03.934319 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:03.934267 2570 prober.go:120] "Probe failed" probeType="Readiness" pod="keycloak-system/maas-keycloak-0" podUID="8f856529-c5c8-4d94-bb26-efd30e4c131f" containerName="keycloak" probeResult="failure" output="HTTP probe failed with statuscode: 503" Jun 02 21:47:13.919512 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:13.919475 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="keycloak-system/maas-keycloak-0" Jun 02 21:47:24.684763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.684726 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:47:24.688038 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.688021 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.693105 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.693083 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"authorino-oidc-ca\"" Jun 02 21:47:24.700770 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.700744 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:47:24.800938 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.800908 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-d72jm\" (UniqueName: \"kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.801116 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.800964 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.801116 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.800985 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.901799 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.901774 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.901892 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.901814 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-d72jm\" (UniqueName: \"kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.901892 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.901857 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.902445 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.902425 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.904144 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.904128 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.913001 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.912982 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-d72jm\" (UniqueName: \"kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm\") pod \"authorino-678698bc95-n7lg2\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:24.996336 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:24.996313 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:47:25.125808 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.125787 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:47:25.128098 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:47:25.128067 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod5758e0ba_8817_41ae_b4c3_0e39cfede205.slice/crio-556dcbb12efce6e09c938a8c7cfade317630298af71e143fce1cae6494452612 WatchSource:0}: Error finding container 556dcbb12efce6e09c938a8c7cfade317630298af71e143fce1cae6494452612: Status 404 returned error can't find the container with id 556dcbb12efce6e09c938a8c7cfade317630298af71e143fce1cae6494452612 Jun 02 21:47:25.568660 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.568629 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678698bc95-n7lg2" event={"ID":"5758e0ba-8817-41ae-b4c3-0e39cfede205","Type":"ContainerStarted","Data":"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220"} Jun 02 21:47:25.568757 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.568672 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678698bc95-n7lg2" event={"ID":"5758e0ba-8817-41ae-b4c3-0e39cfede205","Type":"ContainerStarted","Data":"556dcbb12efce6e09c938a8c7cfade317630298af71e143fce1cae6494452612"} Jun 02 21:47:25.642750 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.642689 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-678698bc95-n7lg2" podStartSLOduration=1.275366662 podStartE2EDuration="1.642668262s" podCreationTimestamp="2026-06-02 21:47:24 +0000 UTC" firstStartedPulling="2026-06-02 21:47:25.129334066 +0000 UTC m=+834.555894545" lastFinishedPulling="2026-06-02 21:47:25.496635674 +0000 UTC m=+834.923196145" observedRunningTime="2026-06-02 21:47:25.603134346 +0000 UTC m=+835.029694834" watchObservedRunningTime="2026-06-02 21:47:25.642668262 +0000 UTC m=+835.069228753" Jun 02 21:47:25.642916 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.642890 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:47:25.643168 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.643134 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-8cf67d99c-ms876" podUID="38846959-50a3-4b32-a2c4-5e295b77e74c" containerName="authorino" containerID="cri-o://c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810" gracePeriod=30 Jun 02 21:47:25.905687 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:25.905601 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:47:26.013326 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.013223 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert\") pod \"38846959-50a3-4b32-a2c4-5e295b77e74c\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " Jun 02 21:47:26.013326 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.013296 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tdsvd\" (UniqueName: \"kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd\") pod \"38846959-50a3-4b32-a2c4-5e295b77e74c\" (UID: \"38846959-50a3-4b32-a2c4-5e295b77e74c\") " Jun 02 21:47:26.015929 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.015891 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd" (OuterVolumeSpecName: "kube-api-access-tdsvd") pod "38846959-50a3-4b32-a2c4-5e295b77e74c" (UID: "38846959-50a3-4b32-a2c4-5e295b77e74c"). InnerVolumeSpecName "kube-api-access-tdsvd". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:47:26.026574 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.026523 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "38846959-50a3-4b32-a2c4-5e295b77e74c" (UID: "38846959-50a3-4b32-a2c4-5e295b77e74c"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:47:26.114241 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.114203 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/38846959-50a3-4b32-a2c4-5e295b77e74c-tls-cert\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:47:26.114241 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.114237 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tdsvd\" (UniqueName: \"kubernetes.io/projected/38846959-50a3-4b32-a2c4-5e295b77e74c-kube-api-access-tdsvd\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:47:26.572760 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.572725 2570 generic.go:358] "Generic (PLEG): container finished" podID="38846959-50a3-4b32-a2c4-5e295b77e74c" containerID="c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810" exitCode=0 Jun 02 21:47:26.572941 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.572771 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-8cf67d99c-ms876" Jun 02 21:47:26.572941 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.572808 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8cf67d99c-ms876" event={"ID":"38846959-50a3-4b32-a2c4-5e295b77e74c","Type":"ContainerDied","Data":"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810"} Jun 02 21:47:26.572941 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.572848 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-8cf67d99c-ms876" event={"ID":"38846959-50a3-4b32-a2c4-5e295b77e74c","Type":"ContainerDied","Data":"6fe5a1d8cfb98792a7edaf80f7ef6dbd5f21e41abeb8aad0e2c5979b847a6bb8"} Jun 02 21:47:26.572941 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.572865 2570 scope.go:117] "RemoveContainer" containerID="c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810" Jun 02 21:47:26.581098 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.581084 2570 scope.go:117] "RemoveContainer" containerID="c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810" Jun 02 21:47:26.581363 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:47:26.581339 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810\": container with ID starting with c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810 not found: ID does not exist" containerID="c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810" Jun 02 21:47:26.581455 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.581369 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810"} err="failed to get container status \"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810\": rpc error: code = NotFound desc = could not find container \"c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810\": container with ID starting with c5a61bf4a5a8eff5174a48504d2bec05d6f3ba6c905f828029069e35f398c810 not found: ID does not exist" Jun 02 21:47:26.598526 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.598504 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:47:26.608246 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:26.608224 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-8cf67d99c-ms876"] Jun 02 21:47:27.142433 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:27.142401 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="38846959-50a3-4b32-a2c4-5e295b77e74c" path="/var/lib/kubelet/pods/38846959-50a3-4b32-a2c4-5e295b77e74c/volumes" Jun 02 21:47:42.982565 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.982532 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx"] Jun 02 21:47:42.982931 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.982807 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="38846959-50a3-4b32-a2c4-5e295b77e74c" containerName="authorino" Jun 02 21:47:42.982931 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.982818 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="38846959-50a3-4b32-a2c4-5e295b77e74c" containerName="authorino" Jun 02 21:47:42.982931 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.982879 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="38846959-50a3-4b32-a2c4-5e295b77e74c" containerName="authorino" Jun 02 21:47:42.985718 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.985700 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:42.988443 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.988424 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-h8v7p\"" Jun 02 21:47:42.988683 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.988668 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\"" Jun 02 21:47:42.989573 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.989555 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-simulated-kserve-self-signed-certs\"" Jun 02 21:47:42.989655 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.989589 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\"" Jun 02 21:47:42.995968 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:42.995946 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx"] Jun 02 21:47:43.134876 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.134837 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.134876 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.134879 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.135120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.134897 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pt9\" (UniqueName: \"kubernetes.io/projected/9b55176f-9138-434c-a5ce-5c62399544b8-kube-api-access-j6pt9\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.135120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.134992 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b55176f-9138-434c-a5ce-5c62399544b8-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.135120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.135063 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.135120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.135091 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236399 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236308 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236399 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236347 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236399 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236364 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pt9\" (UniqueName: \"kubernetes.io/projected/9b55176f-9138-434c-a5ce-5c62399544b8-kube-api-access-j6pt9\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236674 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236426 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b55176f-9138-434c-a5ce-5c62399544b8-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236674 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236476 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236674 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236507 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236826 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236793 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-model-cache\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.236961 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236940 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-kserve-provision-location\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.237032 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.236948 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-home\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.238613 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.238592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/9b55176f-9138-434c-a5ce-5c62399544b8-dshm\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.238982 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.238961 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/9b55176f-9138-434c-a5ce-5c62399544b8-tls-certs\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.245330 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.245309 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pt9\" (UniqueName: \"kubernetes.io/projected/9b55176f-9138-434c-a5ce-5c62399544b8-kube-api-access-j6pt9\") pod \"e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx\" (UID: \"9b55176f-9138-434c-a5ce-5c62399544b8\") " pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.297295 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.297272 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:47:43.420682 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.420648 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx"] Jun 02 21:47:43.423167 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:47:43.423141 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9b55176f_9138_434c_a5ce_5c62399544b8.slice/crio-4aa470dc6b983a745013a6a1f97f87eeb27f651883e6eadb9ed29f91c1f2ce81 WatchSource:0}: Error finding container 4aa470dc6b983a745013a6a1f97f87eeb27f651883e6eadb9ed29f91c1f2ce81: Status 404 returned error can't find the container with id 4aa470dc6b983a745013a6a1f97f87eeb27f651883e6eadb9ed29f91c1f2ce81 Jun 02 21:47:43.629218 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:43.629138 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" event={"ID":"9b55176f-9138-434c-a5ce-5c62399544b8","Type":"ContainerStarted","Data":"4aa470dc6b983a745013a6a1f97f87eeb27f651883e6eadb9ed29f91c1f2ce81"} Jun 02 21:47:46.640078 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:47:46.640026 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" event={"ID":"9b55176f-9138-434c-a5ce-5c62399544b8","Type":"ContainerStarted","Data":"6224d77e3f8bde9f98892bf135fab20fe5953bb9f193e37f9f07bc9166d0b876"} Jun 02 21:48:11.197242 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.197159 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4"] Jun 02 21:48:11.219163 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.219137 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4"] Jun 02 21:48:11.219290 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.219279 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.221707 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.221684 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-unab60ef4d3a239b5143b412cab04acac3-kserve-self-signed-certs\"" Jun 02 21:48:11.337289 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337248 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.337289 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337288 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.337506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337367 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.337506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337463 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8sxwd\" (UniqueName: \"kubernetes.io/projected/6124c65a-aa75-42f1-9269-1eedc366cdcd-kube-api-access-8sxwd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.337506 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337503 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6124c65a-aa75-42f1-9269-1eedc366cdcd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.337618 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.337529 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438373 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438333 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6124c65a-aa75-42f1-9269-1eedc366cdcd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438547 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438384 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438547 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438436 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438547 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438467 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438547 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438518 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438761 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438563 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8sxwd\" (UniqueName: \"kubernetes.io/projected/6124c65a-aa75-42f1-9269-1eedc366cdcd-kube-api-access-8sxwd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438834 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438806 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-model-cache\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438921 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438870 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-kserve-provision-location\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.438921 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.438895 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-home\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.440762 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.440740 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/6124c65a-aa75-42f1-9269-1eedc366cdcd-dshm\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.441003 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.440987 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/6124c65a-aa75-42f1-9269-1eedc366cdcd-tls-certs\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.445725 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.445705 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8sxwd\" (UniqueName: \"kubernetes.io/projected/6124c65a-aa75-42f1-9269-1eedc366cdcd-kube-api-access-8sxwd\") pod \"e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4\" (UID: \"6124c65a-aa75-42f1-9269-1eedc366cdcd\") " pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.537492 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.537457 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:11.662273 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.662252 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4"] Jun 02 21:48:11.664389 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:48:11.664364 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6124c65a_aa75_42f1_9269_1eedc366cdcd.slice/crio-ee74c9c7bd51d364b610005f532d9c774f0fbd2e3fcd91c7fad7e8fec44a7e05 WatchSource:0}: Error finding container ee74c9c7bd51d364b610005f532d9c774f0fbd2e3fcd91c7fad7e8fec44a7e05: Status 404 returned error can't find the container with id ee74c9c7bd51d364b610005f532d9c774f0fbd2e3fcd91c7fad7e8fec44a7e05 Jun 02 21:48:11.717730 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:11.717698 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" event={"ID":"6124c65a-aa75-42f1-9269-1eedc366cdcd","Type":"ContainerStarted","Data":"ee74c9c7bd51d364b610005f532d9c774f0fbd2e3fcd91c7fad7e8fec44a7e05"} Jun 02 21:48:12.723007 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:12.722971 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" event={"ID":"6124c65a-aa75-42f1-9269-1eedc366cdcd","Type":"ContainerStarted","Data":"392330cbb3611b67933a89864e32f5454e5b8c47c25bea54ba6afe3a4ab41b2f"} Jun 02 21:48:13.727391 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:13.727354 2570 generic.go:358] "Generic (PLEG): container finished" podID="6124c65a-aa75-42f1-9269-1eedc366cdcd" containerID="392330cbb3611b67933a89864e32f5454e5b8c47c25bea54ba6afe3a4ab41b2f" exitCode=0 Jun 02 21:48:13.727780 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:13.727450 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" event={"ID":"6124c65a-aa75-42f1-9269-1eedc366cdcd","Type":"ContainerDied","Data":"392330cbb3611b67933a89864e32f5454e5b8c47c25bea54ba6afe3a4ab41b2f"} Jun 02 21:48:15.734860 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:15.734824 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" event={"ID":"6124c65a-aa75-42f1-9269-1eedc366cdcd","Type":"ContainerStarted","Data":"e1e19e57292dd59fc15e0bb9b624abbf507af08dabe20f88ec6f1700e8cad7a8"} Jun 02 21:48:15.751950 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:15.751904 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" podStartSLOduration=3.72101348 podStartE2EDuration="4.751890921s" podCreationTimestamp="2026-06-02 21:48:11 +0000 UTC" firstStartedPulling="2026-06-02 21:48:13.727925867 +0000 UTC m=+883.154486334" lastFinishedPulling="2026-06-02 21:48:14.758803309 +0000 UTC m=+884.185363775" observedRunningTime="2026-06-02 21:48:15.750509794 +0000 UTC m=+885.177070282" watchObservedRunningTime="2026-06-02 21:48:15.751890921 +0000 UTC m=+885.178451446" Jun 02 21:48:21.538459 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:21.538412 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:21.538938 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:21.538575 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:21.551257 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:21.551233 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:21.763472 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:21.763444 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4" Jun 02 21:48:31.095082 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.095058 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:48:31.096756 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.096734 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:48:31.871968 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.871940 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb"] Jun 02 21:48:31.877082 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.877063 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:31.879393 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.879371 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\"" Jun 02 21:48:31.883790 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:31.883765 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb"] Jun 02 21:48:32.008787 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008756 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.008942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008795 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fecca602-d9bf-4331-9e5e-e38a35cf0b14-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.008942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008830 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mwj2t\" (UniqueName: \"kubernetes.io/projected/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kube-api-access-mwj2t\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.008942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008861 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.008942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008886 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.008942 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.008902 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110110 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110079 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fecca602-d9bf-4331-9e5e-e38a35cf0b14-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110482 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110129 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mwj2t\" (UniqueName: \"kubernetes.io/projected/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kube-api-access-mwj2t\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110482 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110154 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110482 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110182 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110482 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110199 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110482 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110235 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110592 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110645 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-home\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.110763 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.110728 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-model-cache\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.112405 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.112378 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fecca602-d9bf-4331-9e5e-e38a35cf0b14-dshm\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.112668 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.112651 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fecca602-d9bf-4331-9e5e-e38a35cf0b14-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.117278 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.117246 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mwj2t\" (UniqueName: \"kubernetes.io/projected/fecca602-d9bf-4331-9e5e-e38a35cf0b14-kube-api-access-mwj2t\") pod \"premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb\" (UID: \"fecca602-d9bf-4331-9e5e-e38a35cf0b14\") " pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.187587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.187540 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:32.308273 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.308251 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb"] Jun 02 21:48:32.310154 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:48:32.310117 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfecca602_d9bf_4331_9e5e_e38a35cf0b14.slice/crio-b900817d659c04fec41438d98f1a2b797c0b0529953db0c02177964bdd1bbb8b WatchSource:0}: Error finding container b900817d659c04fec41438d98f1a2b797c0b0529953db0c02177964bdd1bbb8b: Status 404 returned error can't find the container with id b900817d659c04fec41438d98f1a2b797c0b0529953db0c02177964bdd1bbb8b Jun 02 21:48:32.786271 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.786234 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" event={"ID":"fecca602-d9bf-4331-9e5e-e38a35cf0b14","Type":"ContainerStarted","Data":"84f791ba9b452ee167990accb42d45536d24c02dc7c67199adeb653945bacd57"} Jun 02 21:48:32.786434 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:32.786281 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" event={"ID":"fecca602-d9bf-4331-9e5e-e38a35cf0b14","Type":"ContainerStarted","Data":"b900817d659c04fec41438d98f1a2b797c0b0529953db0c02177964bdd1bbb8b"} Jun 02 21:48:36.802292 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:36.802259 2570 generic.go:358] "Generic (PLEG): container finished" podID="fecca602-d9bf-4331-9e5e-e38a35cf0b14" containerID="84f791ba9b452ee167990accb42d45536d24c02dc7c67199adeb653945bacd57" exitCode=0 Jun 02 21:48:36.802617 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:36.802332 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" event={"ID":"fecca602-d9bf-4331-9e5e-e38a35cf0b14","Type":"ContainerDied","Data":"84f791ba9b452ee167990accb42d45536d24c02dc7c67199adeb653945bacd57"} Jun 02 21:48:37.806539 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:37.806504 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" event={"ID":"fecca602-d9bf-4331-9e5e-e38a35cf0b14","Type":"ContainerStarted","Data":"9ecc84a304f54236757f545186be346d813fc68a10fd62e37c98b70b5f7d337a"} Jun 02 21:48:37.823170 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:37.823102 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" podStartSLOduration=6.635041963 podStartE2EDuration="6.82308592s" podCreationTimestamp="2026-06-02 21:48:31 +0000 UTC" firstStartedPulling="2026-06-02 21:48:36.802879341 +0000 UTC m=+906.229439808" lastFinishedPulling="2026-06-02 21:48:36.990923284 +0000 UTC m=+906.417483765" observedRunningTime="2026-06-02 21:48:37.822771538 +0000 UTC m=+907.249332041" watchObservedRunningTime="2026-06-02 21:48:37.82308592 +0000 UTC m=+907.249646410" Jun 02 21:48:40.818375 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:40.818345 2570 generic.go:358] "Generic (PLEG): container finished" podID="9b55176f-9138-434c-a5ce-5c62399544b8" containerID="6224d77e3f8bde9f98892bf135fab20fe5953bb9f193e37f9f07bc9166d0b876" exitCode=0 Jun 02 21:48:40.818886 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:40.818425 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" event={"ID":"9b55176f-9138-434c-a5ce-5c62399544b8","Type":"ContainerDied","Data":"6224d77e3f8bde9f98892bf135fab20fe5953bb9f193e37f9f07bc9166d0b876"} Jun 02 21:48:41.822708 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:41.822680 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" event={"ID":"9b55176f-9138-434c-a5ce-5c62399544b8","Type":"ContainerStarted","Data":"b486482474ad4cf402873668b16f14e06844878f36fa91ca61bd650079eb79da"} Jun 02 21:48:41.841195 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:41.841153 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" podStartSLOduration=2.229087205 podStartE2EDuration="59.841137266s" podCreationTimestamp="2026-06-02 21:47:42 +0000 UTC" firstStartedPulling="2026-06-02 21:47:43.425007112 +0000 UTC m=+852.851567578" lastFinishedPulling="2026-06-02 21:48:41.037057158 +0000 UTC m=+910.463617639" observedRunningTime="2026-06-02 21:48:41.840111938 +0000 UTC m=+911.266672425" watchObservedRunningTime="2026-06-02 21:48:41.841137266 +0000 UTC m=+911.267697757" Jun 02 21:48:42.188032 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:42.187970 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:42.188032 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:42.187998 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:42.200217 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:42.200195 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:42.838279 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:42.838246 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb" Jun 02 21:48:43.297703 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:43.297666 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:48:43.297703 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:43.297707 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:48:43.310214 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:43.310190 2570 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:48:43.840886 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:48:43.840859 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx" Jun 02 21:49:33.192903 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.192875 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-75db76f69d-w69pw"] Jun 02 21:49:33.194660 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.194645 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.204395 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.204369 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75db76f69d-w69pw"] Jun 02 21:49:33.243286 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.243262 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/78675d29-d65c-447c-b79c-6b145a1cbba1-oidc-ca\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.243382 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.243307 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78675d29-d65c-447c-b79c-6b145a1cbba1-tls-cert\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.243439 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.243419 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q5dwj\" (UniqueName: \"kubernetes.io/projected/78675d29-d65c-447c-b79c-6b145a1cbba1-kube-api-access-q5dwj\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.344553 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.344528 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q5dwj\" (UniqueName: \"kubernetes.io/projected/78675d29-d65c-447c-b79c-6b145a1cbba1-kube-api-access-q5dwj\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.344653 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.344567 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/78675d29-d65c-447c-b79c-6b145a1cbba1-oidc-ca\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.344653 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.344591 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78675d29-d65c-447c-b79c-6b145a1cbba1-tls-cert\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.345120 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.345095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/78675d29-d65c-447c-b79c-6b145a1cbba1-oidc-ca\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.346824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.346805 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/78675d29-d65c-447c-b79c-6b145a1cbba1-tls-cert\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.352516 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.352495 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q5dwj\" (UniqueName: \"kubernetes.io/projected/78675d29-d65c-447c-b79c-6b145a1cbba1-kube-api-access-q5dwj\") pod \"authorino-75db76f69d-w69pw\" (UID: \"78675d29-d65c-447c-b79c-6b145a1cbba1\") " pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.503830 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.503802 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-75db76f69d-w69pw" Jun 02 21:49:33.628073 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.627997 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-75db76f69d-w69pw"] Jun 02 21:49:33.630193 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:49:33.630159 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod78675d29_d65c_447c_b79c_6b145a1cbba1.slice/crio-973d9d6a2b69bc5b5f98b9732f42c01ac005e882f5569b66a3316f7dbf880f43 WatchSource:0}: Error finding container 973d9d6a2b69bc5b5f98b9732f42c01ac005e882f5569b66a3316f7dbf880f43: Status 404 returned error can't find the container with id 973d9d6a2b69bc5b5f98b9732f42c01ac005e882f5569b66a3316f7dbf880f43 Jun 02 21:49:33.989467 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.989434 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75db76f69d-w69pw" event={"ID":"78675d29-d65c-447c-b79c-6b145a1cbba1","Type":"ContainerStarted","Data":"dc72c50dc8465f7d68f0f39bd7b4780132a1f32b7a4dd31a40fe1b3a5a2ee9bc"} Jun 02 21:49:33.989666 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:33.989477 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-75db76f69d-w69pw" event={"ID":"78675d29-d65c-447c-b79c-6b145a1cbba1","Type":"ContainerStarted","Data":"973d9d6a2b69bc5b5f98b9732f42c01ac005e882f5569b66a3316f7dbf880f43"} Jun 02 21:49:34.006595 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.006553 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-75db76f69d-w69pw" podStartSLOduration=0.749564237 podStartE2EDuration="1.006539311s" podCreationTimestamp="2026-06-02 21:49:33 +0000 UTC" firstStartedPulling="2026-06-02 21:49:33.631548284 +0000 UTC m=+963.058108750" lastFinishedPulling="2026-06-02 21:49:33.888523358 +0000 UTC m=+963.315083824" observedRunningTime="2026-06-02 21:49:34.005461446 +0000 UTC m=+963.432021936" watchObservedRunningTime="2026-06-02 21:49:34.006539311 +0000 UTC m=+963.433099799" Jun 02 21:49:34.033480 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.033447 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:49:34.033728 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.033699 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-678698bc95-n7lg2" podUID="5758e0ba-8817-41ae-b4c3-0e39cfede205" containerName="authorino" containerID="cri-o://f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220" gracePeriod=30 Jun 02 21:49:34.280859 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.280835 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:49:34.352991 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.352951 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-d72jm\" (UniqueName: \"kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm\") pod \"5758e0ba-8817-41ae-b4c3-0e39cfede205\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " Jun 02 21:49:34.352991 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.352987 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert\") pod \"5758e0ba-8817-41ae-b4c3-0e39cfede205\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " Jun 02 21:49:34.353243 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.353031 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca\") pod \"5758e0ba-8817-41ae-b4c3-0e39cfede205\" (UID: \"5758e0ba-8817-41ae-b4c3-0e39cfede205\") " Jun 02 21:49:34.356895 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.356510 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm" (OuterVolumeSpecName: "kube-api-access-d72jm") pod "5758e0ba-8817-41ae-b4c3-0e39cfede205" (UID: "5758e0ba-8817-41ae-b4c3-0e39cfede205"). InnerVolumeSpecName "kube-api-access-d72jm". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:49:34.358731 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.358651 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca" (OuterVolumeSpecName: "oidc-ca") pod "5758e0ba-8817-41ae-b4c3-0e39cfede205" (UID: "5758e0ba-8817-41ae-b4c3-0e39cfede205"). InnerVolumeSpecName "oidc-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue "" Jun 02 21:49:34.364262 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.364237 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert" (OuterVolumeSpecName: "tls-cert") pod "5758e0ba-8817-41ae-b4c3-0e39cfede205" (UID: "5758e0ba-8817-41ae-b4c3-0e39cfede205"). InnerVolumeSpecName "tls-cert". PluginName "kubernetes.io/secret", VolumeGIDValue "" Jun 02 21:49:34.454293 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.454220 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-d72jm\" (UniqueName: \"kubernetes.io/projected/5758e0ba-8817-41ae-b4c3-0e39cfede205-kube-api-access-d72jm\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:49:34.454293 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.454248 2570 reconciler_common.go:299] "Volume detached for volume \"tls-cert\" (UniqueName: \"kubernetes.io/secret/5758e0ba-8817-41ae-b4c3-0e39cfede205-tls-cert\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:49:34.454293 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.454259 2570 reconciler_common.go:299] "Volume detached for volume \"oidc-ca\" (UniqueName: \"kubernetes.io/configmap/5758e0ba-8817-41ae-b4c3-0e39cfede205-oidc-ca\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:49:34.996062 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.996011 2570 generic.go:358] "Generic (PLEG): container finished" podID="5758e0ba-8817-41ae-b4c3-0e39cfede205" containerID="f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220" exitCode=0 Jun 02 21:49:34.996216 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.996093 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-678698bc95-n7lg2" Jun 02 21:49:34.996216 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.996088 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678698bc95-n7lg2" event={"ID":"5758e0ba-8817-41ae-b4c3-0e39cfede205","Type":"ContainerDied","Data":"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220"} Jun 02 21:49:34.996216 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.996131 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-678698bc95-n7lg2" event={"ID":"5758e0ba-8817-41ae-b4c3-0e39cfede205","Type":"ContainerDied","Data":"556dcbb12efce6e09c938a8c7cfade317630298af71e143fce1cae6494452612"} Jun 02 21:49:34.996216 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:34.996145 2570 scope.go:117] "RemoveContainer" containerID="f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220" Jun 02 21:49:35.004344 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:35.004296 2570 scope.go:117] "RemoveContainer" containerID="f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220" Jun 02 21:49:35.004623 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:49:35.004594 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220\": container with ID starting with f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220 not found: ID does not exist" containerID="f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220" Jun 02 21:49:35.004721 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:35.004633 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220"} err="failed to get container status \"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220\": rpc error: code = NotFound desc = could not find container \"f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220\": container with ID starting with f66debe006a9fe08f9e93c53c66e3daa966611f5d777ff59c3d0cea337a64220 not found: ID does not exist" Jun 02 21:49:35.016927 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:35.016904 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:49:35.021913 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:35.021890 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-678698bc95-n7lg2"] Jun 02 21:49:35.141054 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:49:35.141016 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5758e0ba-8817-41ae-b4c3-0e39cfede205" path="/var/lib/kubelet/pods/5758e0ba-8817-41ae-b4c3-0e39cfede205/volumes" Jun 02 21:51:03.181094 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.181058 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:51:03.181605 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.181302 2570 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" podUID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" containerName="manager" containerID="cri-o://8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de" gracePeriod=30 Jun 02 21:51:03.429466 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.429440 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:51:03.535267 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.535231 2570 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sglh4\" (UniqueName: \"kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4\") pod \"56001ee8-ef73-4855-bfb1-0faf8fc4218d\" (UID: \"56001ee8-ef73-4855-bfb1-0faf8fc4218d\") " Jun 02 21:51:03.537259 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.537234 2570 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4" (OuterVolumeSpecName: "kube-api-access-sglh4") pod "56001ee8-ef73-4855-bfb1-0faf8fc4218d" (UID: "56001ee8-ef73-4855-bfb1-0faf8fc4218d"). InnerVolumeSpecName "kube-api-access-sglh4". PluginName "kubernetes.io/projected", VolumeGIDValue "" Jun 02 21:51:03.636392 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:03.636350 2570 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sglh4\" (UniqueName: \"kubernetes.io/projected/56001ee8-ef73-4855-bfb1-0faf8fc4218d-kube-api-access-sglh4\") on node \"ip-10-0-134-230.ec2.internal\" DevicePath \"\"" Jun 02 21:51:04.286123 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.286084 2570 generic.go:358] "Generic (PLEG): container finished" podID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" containerID="8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de" exitCode=0 Jun 02 21:51:04.286587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.286152 2570 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" Jun 02 21:51:04.286587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.286169 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" event={"ID":"56001ee8-ef73-4855-bfb1-0faf8fc4218d","Type":"ContainerDied","Data":"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de"} Jun 02 21:51:04.286587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.286209 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-lnlcf" event={"ID":"56001ee8-ef73-4855-bfb1-0faf8fc4218d","Type":"ContainerDied","Data":"f6ea7f5268a108d2afa7fa5128e93f7eed1cc4592680d88856383c64ba4b6627"} Jun 02 21:51:04.286587 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.286223 2570 scope.go:117] "RemoveContainer" containerID="8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de" Jun 02 21:51:04.294207 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.293988 2570 scope.go:117] "RemoveContainer" containerID="8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de" Jun 02 21:51:04.294420 ip-10-0-134-230 kubenswrapper[2570]: E0602 21:51:04.294397 2570 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de\": container with ID starting with 8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de not found: ID does not exist" containerID="8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de" Jun 02 21:51:04.294510 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.294426 2570 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de"} err="failed to get container status \"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de\": rpc error: code = NotFound desc = could not find container \"8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de\": container with ID starting with 8a74eeed924b61f975f6bc2e45a9560339816144ee11083c50043d71ed9bc2de not found: ID does not exist" Jun 02 21:51:04.305851 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.305826 2570 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:51:04.309429 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:04.309407 2570 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-lnlcf"] Jun 02 21:51:05.024862 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.024832 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-82gxp"] Jun 02 21:51:05.025161 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025149 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5758e0ba-8817-41ae-b4c3-0e39cfede205" containerName="authorino" Jun 02 21:51:05.025212 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025163 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="5758e0ba-8817-41ae-b4c3-0e39cfede205" containerName="authorino" Jun 02 21:51:05.025212 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025177 2570 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" containerName="manager" Jun 02 21:51:05.025212 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025182 2570 state_mem.go:107] "Deleted CPUSet assignment" podUID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" containerName="manager" Jun 02 21:51:05.025305 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025238 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" containerName="manager" Jun 02 21:51:05.025305 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.025247 2570 memory_manager.go:356] "RemoveStaleState removing state" podUID="5758e0ba-8817-41ae-b4c3-0e39cfede205" containerName="authorino" Jun 02 21:51:05.029008 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.028986 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:05.031136 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.031110 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\"" Jun 02 21:51:05.031136 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.031136 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-2xv5f\"" Jun 02 21:51:05.033908 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.033887 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-82gxp"] Jun 02 21:51:05.141394 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.141363 2570 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="56001ee8-ef73-4855-bfb1-0faf8fc4218d" path="/var/lib/kubelet/pods/56001ee8-ef73-4855-bfb1-0faf8fc4218d/volumes" Jun 02 21:51:05.150239 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.150211 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bbvmb\" (UniqueName: \"kubernetes.io/projected/8e98df39-f8ec-4eff-aea7-7d73f94a3f81-kube-api-access-bbvmb\") pod \"maas-controller-764f4b5cd5-82gxp\" (UID: \"8e98df39-f8ec-4eff-aea7-7d73f94a3f81\") " pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:05.251626 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.251596 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bbvmb\" (UniqueName: \"kubernetes.io/projected/8e98df39-f8ec-4eff-aea7-7d73f94a3f81-kube-api-access-bbvmb\") pod \"maas-controller-764f4b5cd5-82gxp\" (UID: \"8e98df39-f8ec-4eff-aea7-7d73f94a3f81\") " pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:05.258975 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.258943 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bbvmb\" (UniqueName: \"kubernetes.io/projected/8e98df39-f8ec-4eff-aea7-7d73f94a3f81-kube-api-access-bbvmb\") pod \"maas-controller-764f4b5cd5-82gxp\" (UID: \"8e98df39-f8ec-4eff-aea7-7d73f94a3f81\") " pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:05.340597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.340510 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:05.467457 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:05.467431 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-764f4b5cd5-82gxp"] Jun 02 21:51:05.469623 ip-10-0-134-230 kubenswrapper[2570]: W0602 21:51:05.469598 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8e98df39_f8ec_4eff_aea7_7d73f94a3f81.slice/crio-d57a3a12e0d573df35966f6f356f6bdedccacdb02c33b59754d45cc2bf637612 WatchSource:0}: Error finding container d57a3a12e0d573df35966f6f356f6bdedccacdb02c33b59754d45cc2bf637612: Status 404 returned error can't find the container with id d57a3a12e0d573df35966f6f356f6bdedccacdb02c33b59754d45cc2bf637612 Jun 02 21:51:06.293824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:06.293794 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" event={"ID":"8e98df39-f8ec-4eff-aea7-7d73f94a3f81","Type":"ContainerStarted","Data":"7938cda228781cbe9594e3d7195bea34e80565c061a495e6d203010f94409107"} Jun 02 21:51:06.293824 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:06.293828 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" event={"ID":"8e98df39-f8ec-4eff-aea7-7d73f94a3f81","Type":"ContainerStarted","Data":"d57a3a12e0d573df35966f6f356f6bdedccacdb02c33b59754d45cc2bf637612"} Jun 02 21:51:06.294019 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:06.293912 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:51:06.310597 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:06.310552 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" podStartSLOduration=0.91580825 podStartE2EDuration="1.310541705s" podCreationTimestamp="2026-06-02 21:51:05 +0000 UTC" firstStartedPulling="2026-06-02 21:51:05.471233554 +0000 UTC m=+1054.897794023" lastFinishedPulling="2026-06-02 21:51:05.865967011 +0000 UTC m=+1055.292527478" observedRunningTime="2026-06-02 21:51:06.308125874 +0000 UTC m=+1055.734686361" watchObservedRunningTime="2026-06-02 21:51:06.310541705 +0000 UTC m=+1055.737102192" Jun 02 21:51:17.303150 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:51:17.303119 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-764f4b5cd5-82gxp" Jun 02 21:53:31.115977 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:53:31.115938 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:53:31.118131 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:53:31.118105 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:58:31.135791 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:58:31.135762 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 21:58:31.140309 ip-10-0-134-230 kubenswrapper[2570]: I0602 21:58:31.140286 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:03:31.156802 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:03:31.156774 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:03:31.167447 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:03:31.167426 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:08:31.176948 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:08:31.176920 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:08:31.194329 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:08:31.194309 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:12:38.659505 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:38.659478 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db76f69d-w69pw_78675d29-d65c-447c-b79c-6b145a1cbba1/authorino/0.log" Jun 02 22:12:43.705303 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:43.705266 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6bcd9cbcc4-dss74_9c723b92-d186-4e78-a2c1-0181d7e84dd6/maas-api/0.log" Jun 02 22:12:43.937258 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:43.937228 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764f4b5cd5-82gxp_8e98df39-f8ec-4eff-aea7-7d73f94a3f81/manager/0.log" Jun 02 22:12:44.177268 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:44.177201 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-s5g6q_7c618835-b03f-49ab-a49c-c23c2d710d45/manager/0.log" Jun 02 22:12:44.536398 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:44.536368 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-sgfrv_0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1/copy-manifests/0.log" Jun 02 22:12:44.547864 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:44.547840 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-sgfrv_0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1/manager/0.log" Jun 02 22:12:45.935887 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:45.935856 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db76f69d-w69pw_78675d29-d65c-447c-b79c-6b145a1cbba1/authorino/0.log" Jun 02 22:12:46.269290 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:46.269265 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-bkq8v_87c14b9c-833b-4c4a-b50b-584d0d5570d5/kuadrant-console-plugin/0.log" Jun 02 22:12:47.293301 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:47.293272 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcdbb48fb-jbf5m_0d338093-f371-4091-b33f-b7c26b4baf05/kube-auth-proxy/0.log" Jun 02 22:12:48.175057 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.175026 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx_9b55176f-9138-434c-a5ce-5c62399544b8/storage-initializer/0.log" Jun 02 22:12:48.181628 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.181606 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-simulated-kserve-5489f4f4bc-lwwqx_9b55176f-9138-434c-a5ce-5c62399544b8/main/0.log" Jun 02 22:12:48.411873 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.411844 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4_6124c65a-aa75-42f1-9269-1eedc366cdcd/storage-initializer/0.log" Jun 02 22:12:48.418643 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.418622 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-unconfigured-facebook-opt-125m-simulated-kserve-bdd7f9x6wd4_6124c65a-aa75-42f1-9269-1eedc366cdcd/main/0.log" Jun 02 22:12:48.643764 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.643740 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb_fecca602-d9bf-4331-9e5e-e38a35cf0b14/storage-initializer/0.log" Jun 02 22:12:48.649952 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:48.649930 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-d8bdbcb5c-jsdlb_fecca602-d9bf-4331-9e5e-e38a35cf0b14/main/0.log" Jun 02 22:12:56.020266 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:56.020233 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-mz9vm_81fe2e5d-3e81-4d24-b43e-52ab19e21e86/global-pull-secret-syncer/0.log" Jun 02 22:12:56.136179 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:56.136146 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-7w995_10581f86-1587-4d24-97fc-7630c1309315/konnectivity-agent/0.log" Jun 02 22:12:56.240370 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:12:56.240343 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-134-230.ec2.internal_51bf9710c49ae2c5fd873f7bcc330dcd/haproxy/0.log" Jun 02 22:13:00.435928 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:00.435900 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-75db76f69d-w69pw_78675d29-d65c-447c-b79c-6b145a1cbba1/authorino/0.log" Jun 02 22:13:00.513194 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:00.513170 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-bkq8v_87c14b9c-833b-4c4a-b50b-584d0d5570d5/kuadrant-console-plugin/0.log" Jun 02 22:13:02.486876 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:02.486848 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4klbf_c1238f0d-d5bd-478e-9b46-0bd7f1d575fb/node-exporter/0.log" Jun 02 22:13:02.510875 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:02.510855 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4klbf_c1238f0d-d5bd-478e-9b46-0bd7f1d575fb/kube-rbac-proxy/0.log" Jun 02 22:13:02.536800 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:02.536784 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-4klbf_c1238f0d-d5bd-478e-9b46-0bd7f1d575fb/init-textfile/0.log" Jun 02 22:13:04.645647 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:04.645618 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-76c9bb5fdf-k5r9k_9aa85708-a571-4b8c-862b-f77475fb5a2a/networking-console-plugin/0.log" Jun 02 22:13:07.133678 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.133649 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pnh9b_994dfd6d-3332-447f-869d-f4353d792e2c/dns/0.log" Jun 02 22:13:07.151889 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.151868 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-pnh9b_994dfd6d-3332-447f-869d-f4353d792e2c/kube-rbac-proxy/0.log" Jun 02 22:13:07.171102 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.171083 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-bsngs_cdf0cb33-6a7d-4973-8c72-6292c5710ef9/dns-node-resolver/0.log" Jun 02 22:13:07.303548 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.303528 2570 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff"] Jun 02 22:13:07.306558 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.306545 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.308875 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.308856 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6k8gj\"/\"openshift-service-ca.crt\"" Jun 02 22:13:07.309775 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.309756 2570 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6k8gj\"/\"default-dockercfg-77j6s\"" Jun 02 22:13:07.309775 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.309770 2570 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6k8gj\"/\"kube-root-ca.crt\"" Jun 02 22:13:07.317770 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.313081 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff"] Jun 02 22:13:07.355097 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.355076 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-podres\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.355193 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.355124 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-sys\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.355193 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.355154 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-lib-modules\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.355276 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.355197 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rtbzb\" (UniqueName: \"kubernetes.io/projected/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-kube-api-access-rtbzb\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.355276 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.355233 2570 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-proc\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455820 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455798 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-sys\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455908 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455840 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-lib-modules\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455908 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455865 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rtbzb\" (UniqueName: \"kubernetes.io/projected/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-kube-api-access-rtbzb\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455994 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455915 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-sys\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455994 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455952 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-proc\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.455994 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455989 2570 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-podres\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.456164 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.455997 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-lib-modules\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.456164 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.456095 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-proc\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.456258 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.456176 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-podres\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.471860 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.471830 2570 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rtbzb\" (UniqueName: \"kubernetes.io/projected/f3eca6da-a44c-4688-bbe7-6b6945bc5ec8-kube-api-access-rtbzb\") pod \"perf-node-gather-daemonset-nqvff\" (UID: \"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8\") " pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.623320 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.623299 2570 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:07.746740 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.746711 2570 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff"] Jun 02 22:13:07.749214 ip-10-0-134-230 kubenswrapper[2570]: W0602 22:13:07.749182 2570 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podf3eca6da_a44c_4688_bbe7_6b6945bc5ec8.slice/crio-666d03a9666518711a8d13eefdbf749de6c2962fa0c1f92ad261cc1183e5a35f WatchSource:0}: Error finding container 666d03a9666518711a8d13eefdbf749de6c2962fa0c1f92ad261cc1183e5a35f: Status 404 returned error can't find the container with id 666d03a9666518711a8d13eefdbf749de6c2962fa0c1f92ad261cc1183e5a35f Jun 02 22:13:07.750731 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.750716 2570 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider Jun 02 22:13:07.794739 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:07.794722 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-7t9s9_9afa5d28-b5ca-4780-8f04-89166c9b1054/node-ca/0.log" Jun 02 22:13:08.430134 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:08.430103 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" event={"ID":"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8","Type":"ContainerStarted","Data":"1b2e74deee0e7027fdafbaca8224742db073bd1cb0cc08bd7ed4271ac138d9a0"} Jun 02 22:13:08.430134 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:08.430136 2570 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" event={"ID":"f3eca6da-a44c-4688-bbe7-6b6945bc5ec8","Type":"ContainerStarted","Data":"666d03a9666518711a8d13eefdbf749de6c2962fa0c1f92ad261cc1183e5a35f"} Jun 02 22:13:08.430554 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:08.430233 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:08.446509 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:08.446465 2570 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" podStartSLOduration=1.4464552720000001 podStartE2EDuration="1.446455272s" podCreationTimestamp="2026-06-02 22:13:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-06-02 22:13:08.444414124 +0000 UTC m=+2377.870974612" watchObservedRunningTime="2026-06-02 22:13:08.446455272 +0000 UTC m=+2377.873015761" Jun 02 22:13:08.721169 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:08.721145 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_kube-auth-proxy-dcdbb48fb-jbf5m_0d338093-f371-4091-b33f-b7c26b4baf05/kube-auth-proxy/0.log" Jun 02 22:13:09.392267 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:09.392236 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-gzfbq_68ef55e0-fb04-4198-b200-179452490e9f/serve-healthcheck-canary/0.log" Jun 02 22:13:09.855981 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:09.855953 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcfwd_5ad3af50-c39e-4e8d-8b79-f38ba30aaae4/kube-rbac-proxy/0.log" Jun 02 22:13:09.875773 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:09.875740 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcfwd_5ad3af50-c39e-4e8d-8b79-f38ba30aaae4/exporter/0.log" Jun 02 22:13:09.894890 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:09.894864 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jcfwd_5ad3af50-c39e-4e8d-8b79-f38ba30aaae4/extractor/0.log" Jun 02 22:13:12.186397 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:12.186371 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-api-6bcd9cbcc4-dss74_9c723b92-d186-4e78-a2c1-0181d7e84dd6/maas-api/0.log" Jun 02 22:13:12.274359 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:12.274318 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-764f4b5cd5-82gxp_8e98df39-f8ec-4eff-aea7-7d73f94a3f81/manager/0.log" Jun 02 22:13:12.333722 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:12.333679 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_odh-model-controller-76588b95-s5g6q_7c618835-b03f-49ab-a49c-c23c2d710d45/manager/0.log" Jun 02 22:13:12.508781 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:12.508745 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-sgfrv_0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1/manager/0.log" Jun 02 22:13:12.526501 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:12.526474 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-8b78d96d9-sgfrv_0e532d1d-4a4d-4eb8-870c-4d4f72ea02c1/copy-manifests/0.log" Jun 02 22:13:13.776633 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:13.776605 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-78c4cd8bbd-dlhgp_e1cb1052-6417-4612-b03c-c1bf98655ff6/manager/0.log" Jun 02 22:13:13.822209 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:13.822185 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_openshift-lws-operator-bfc7f696d-p67z5_c89ff155-20e1-42b9-b4ed-7aa065ebd812/openshift-lws-operator/0.log" Jun 02 22:13:14.443209 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:14.443178 2570 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6k8gj/perf-node-gather-daemonset-nqvff" Jun 02 22:13:19.933146 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:19.933116 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/kube-multus-additional-cni-plugins/0.log" Jun 02 22:13:19.951808 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:19.951779 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/egress-router-binary-copy/0.log" Jun 02 22:13:19.969903 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:19.969880 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/cni-plugins/0.log" Jun 02 22:13:19.986896 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:19.986875 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/bond-cni-plugin/0.log" Jun 02 22:13:20.004435 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.004415 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/routeoverride-cni/0.log" Jun 02 22:13:20.022102 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.022084 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/whereabouts-cni-bincopy/0.log" Jun 02 22:13:20.042847 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.042832 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-2dk4v_291f68dc-768f-463f-8177-f9b25147390e/whereabouts-cni/0.log" Jun 02 22:13:20.269301 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.269273 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-wbmzn_76b9dfcd-868b-46a7-8e10-4b624786b63e/kube-multus/0.log" Jun 02 22:13:20.377295 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.377270 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vq7rr_a88961b4-8e81-4ea7-a1df-6bb7c5534afa/network-metrics-daemon/0.log" Jun 02 22:13:20.394310 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:20.394291 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-vq7rr_a88961b4-8e81-4ea7-a1df-6bb7c5534afa/kube-rbac-proxy/0.log" Jun 02 22:13:21.185598 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.185568 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-controller/0.log" Jun 02 22:13:21.200795 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.200764 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/0.log" Jun 02 22:13:21.210748 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.210729 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovn-acl-logging/1.log" Jun 02 22:13:21.230796 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.230763 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/kube-rbac-proxy-node/0.log" Jun 02 22:13:21.249492 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.249468 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/kube-rbac-proxy-ovn-metrics/0.log" Jun 02 22:13:21.266344 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.266319 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/northd/0.log" Jun 02 22:13:21.286919 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.286896 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/nbdb/0.log" Jun 02 22:13:21.304378 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.304360 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/sbdb/0.log" Jun 02 22:13:21.399225 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:21.399189 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-2ltrg_c682cb62-1356-4ee4-ad9a-f7a368bdba49/ovnkube-controller/0.log" Jun 02 22:13:23.002921 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:23.002892 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-scmcl_2f994d2c-9570-4f84-88b2-53c4d15e5cda/network-check-target-container/0.log" Jun 02 22:13:24.006016 ip-10-0-134-230 kubenswrapper[2570]: I0602 22:13:24.005985 2570 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-dmjpb_be0b314d-538c-4868-a86d-72964ae80f4a/iptables-alerter/0.log"